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Executive Summary 


The Technology Immersion Pilot (TIP), created by the Texas Legislature in 2003, was based on the 
assumption that the use of technology in Texas public schools could be achieved more effectively by 
“immersing” schools in technology rather than by introducing technology resources, such as hardware, 
software, digital content, and educator training, in a cyclical fashion over time. The Texas Education 
Agency (TEA) invested more than $20 million in federal Title II, Paid D monies to fund Technology 
Immersion projects at high-need middle schools through a competitive grant process. Concurrently, a 
research study partially funded by a federal Evaluating State Educational Technology Programs grant 
has investigated whether student achievement improved over time through exposure to Technology 
Immersion. The Texas Center for Educational Research (TCER) was TEA’s partner for a four-year 
evaluation of the implementation and effectiveness of the Technology Immersion model. The study 
addressed five major research questions: 

• What was the effect of Technology Immersion on teachers and teaching? 

• What was the effect of Technology Immersion on students and learning? 

• What was the effect of Technology Immersion on students’ academic achievement? 

• How well was Technology Immersion implemented, and 

• What was the relationship between implementation and student academic outcomes? 

Technology Immersion 

State statute described Technology Immersion generally, but to advance consistent interpretation of 
Technology Immersion at schools, the TEA issued a Request for Qualifications for commercial 
vendors to apply to become providers of Technology Immersion packages. Vendors’ plans had to 
include six components: (a) a wireless mobile computing device for each educator and student on an 
immersed campus; (b) productivity, communication, and presentation software; (c) online instructional 
resources supporting the state curriculum in English language arts, mathematics, science, and social 
studies; (d) online assessments to diagnose students’ mastery of the core curriculum; (e) professional 
development designed to help teachers integrate technology into teaching, learning, and the 
curriculum; and (f) initial and ongoing technical support. Through an expert review process, the TEA 
selected three lead vendors to provide Technology immersion packages (Dell Computer Inc., Apple 
Computer Inc., and Region 1 Education Service Center [ESC]). Of the 21 Technology Immersion 
schools studied in the evaluation, 5 middle schools selected the Apple package, 15 selected the Dell 
package, and 1 school selected the Region 1 ESC package (with Dell computers). 

The Theoretical Framework for Technology Immersion guided the evaluation. The framework 
postulated a linear sequence of causal relationships. First, treatment schools were to be “immersed” in 
technology through the introduction of Technology Immersion components. An improved school 
environment for technology was expected to produce teachers who were more technically proficient, 
used technology for professional productivity, had students use technology in their classes, and used 
laptops and digital resources to increase the intellectual rigor of lessons. In turn, changed school and 
classroom conditions were expected to improve students’ technology proficiency, learning 
experiences, collaborative interactions with peers, personal self-direction, and engagement in school 
and learning. Changes in students and their learning experiences presumably contributed to increased 




academic performance as measured by standardized test scores. In the framework, prior student 
achievement and student, family, and school characteristics exerted their own influence on learning. 

Methodology 

The fourth-year evaluation provides final conclusions about the effects of Technology Immersion on 
schools, teachers, and students. This report combines information gathered during the fourth project 
year (2007-08) with data from the first- thro ugh- third implementation years (2004-05 through 2006- 
07). The study’s quasi-experimental research design has allowed inferences about the causal effects of 
Technology Immersion through comparisons between 21 treatment schools and 21 control schools. 

Setting and Participants 

The 42 participating schools included Grades 6 to 8 middle schools drawn from rural, suburban, and 
urban locations across Texas. Middle schools were typically small (about 400 students, on average); 
however, enrollments varied widely (from 83 to 1,447 students). About two-thirds of schools were 
located in small or very small Texas districts (less than 3,000 students), and about a third were in very 
large districts (10,000 or more students). Students in the study were mostly economically 
disadvantaged (67%) and they were racially and ethnically diverse (roughly 58% Hispanic, 7% 

African American, and 36% White). 

The study focused on three student cohorts in the fourth year. Cohort 2 included eighth graders (2,578 
treatment and 2,858 control students) who finished their third immersion year; Cohort 3 included 
seventh graders (2,547 treatment and 2,845 control students) who concluded their second year. We 
also examined achievement data for Cohort 1 students (2,469 treatment students and 2,748 control- 
group students) who had attended Technology Immersion and control schools from sixth-through- 
eighth grade and then attended traditional high schools in the fourth year (high schools that typically 
did not provide individual laptops for students). 

Data Collection and Analysis 

Data came from qualitative and quantitative sources. Researchers conducted site visits at each of the 
middle schools in fall 2004 and again in spring 2005 through 2008. For this report, we concentrated on 
data gathered through observations in a sample of Grades 6, 7, and 8 classrooms (English language 
arts, mathematics, social studies, and science). Additional measures included annual online teacher 
surveys and student paper-and-pencil surveys. We also gathered school and student data on a yearly 
basis from the Texas Public Education Information Management System (PEIMS) and the Academic 
Excellence Indicator System (AEIS), as well as data on student disciplinary actions from individual 
schools. We used three-level hierarchical linear models (HLM) to analyze immersion effects on 
teachers’ and students’ perceptions of their technical proficiencies and technology use, and the effects 
of immersion on students’ Texas Assessment of Knowledge and Skills (TAKS) scores. HLM growth 
modeling estimated the effects of immersion on rates of growth for dependent variables across time 
(2004, 2005, 2006, 2007, and 2008). Two-level HLM models were used to analyze associations 
between the strength of implementation and students’ TAKS achievement. 

Study Limitations 

The study’s quasi-experimental research design had good internal validity given that initially there 
were no statistically significant differences between the treatment and con tied schools. A threat to 
internal validity was introduced in the third and fourth years when control schools began to plan for 
Technology Immersion. Many control teachers received laptops, instructional resources, and intensive 
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professional development in the third year, and in the fourth year, some students at control schools 
received laptops (about 260 or 9% of eighth graders and 480 or 17% of seventh graders). Thus, the 
introduction of Technology Immersion components in control schools may bias fourth-year results. 
Generalization of findings to a broader population (external validity) is another study limitation. 
Compared to Texas middle-school students overall, students in the sample schools were substantially 
more Hispanic and less White and African American. Middle schools were also smaller than the 
statewide average, and schools were located either in small or very small districts or large districts, 
which is different from the statewide distribution of schools. The study also relied on self-reported 
data from surveys of teachers and students, so some findings reflected respondents’ perceptions. 
Nonetheless, the triangulation of evidence from multiple sources (surveys, classroom observations, 
state demographic and test databases, multiple student cohorts) verifies the robustness of findings. 
Researchers are reasonably confident that reported effects can be attributed to the treatment. 

Major Findings 

Like previous years, outcomes represented the effects of Technology Immersion for schools that 
generally reached less than full implementation. Major findings from the fourth year arc described in 
the following sections. A final section discusses the quality of Technology Immersion implementation, 
prospects for sustainability of the model, and implications for educational policy. 

Effects of Technology Immersion on Teachers and Teaching 

Teachers in Technology Immersion schools grew in technology proficiency, their use of 
technology for professional productivity, and their use of technology for student learning 
activities at significantly faster rates than control teachers. Technology Immersion teachers were 
increasingly more technology proficient (i.e., technology operations and pedagogical skills), and they 
used technology more often for professional productivity purposes. However, as control teachers 
acquired more technology resources through immersion grants, differences between teacher groups 
narrowed. Nevertheless, teachers at Technology Immersion schools, who had greater classroom access 
to computers, increased the frequency of their students’ Classroom Activities involving technology at 
a more rapid pace. Fourth-year averages showed that students used a variety of technology resources 
but each of the applications was used infrequently (i.e., about once or twice a month on average). 
Similar to previous years, English language arts, science, and social studies teachers had students use 
technology considerably more often than mathematics teachers. 

Teachers at Technology Immersion schools expressed significantly stronger ideological 
associations across years with technology integration and learner-centered practices. Teachers at 
both immersion and control schools became more positive towards innovative technology practices 
across years, but immersion teachers altered their beliefs at a significantly faster rate. Thus, immersion 
teachers increasingly employed actions such as promoting students’ authentic problem solving or 
critical thinking through technology, and they expressed increasingly stronger affiliations with learner- 
centered practices, such as having students establish individual learning goals and emphasizing 
experiential learning. 

The introduction of Technology Immersion components in schools affected teachers’ perceptions 
of the school’s culture as well as the frequency of teachers’ collegial interactions. Across the first 
two project years, teachers in immersion schools compared to control reported significantly stronger 
leadership for technology, parent and community support, culture of innovation, and collaborative 
interactions with colleagues. Differences between the views of treatment and control teachers 
dissipated in the third and fourth years after introducing technology resources in control schools. 
Control teachers who experienced aspects of Technology Immersion thought their schools’ technology 
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environments were more innovative and supportive. Still, teachers at Technology Immersion schools 
who had access to technology resources over a longer period of time had more frequent collaborative 
interactions with their colleagues that supported instructional practices involving technology. 

Evidence from classroom observations suggested that laptop computers and digital resources 
allowed students in Technology Immersion schools to experience somewhat more intellectually 
demanding work. Observations of core-subject classes (English language arts/reading, mathematics, 
science, and social studies) revealed no statistically significant differences between the overall 
Intellectual Challenge of treatment and control teachers’ instruction. However, effect sizes measuring 
instructional differences between groups generally showed positive effects favoring immersion 
teachers, especially for the domains measuring Higher Order Thinking (e.g., synthesizing, 
generalizing, explaining) and Depth of Knowledge (e.g., thorough exploration of a topic that produces 
complex understandings). Nevertheless, results for all observed teachers indicated that lessons in 
middle school core-subject classes generally failed to intellectually challenge students, with average 
fourth-year ratings of about 2 on the 5-point intellectual challenge scale. 

Effects of Technology Immersion on Students and Learning 

Economically advantaged and disadvantaged students in Technology Immersion schools became 
significantly more technology proficient than their counterparts in control schools. Economically 
disadvantaged immersion students reached proficiency levels that matched the skills of 
advantaged control students. Across implementation years and cohorts, students in Technology 
Immersion schools have made significantly greater progress in mastering the Texas Technology 
Applications standards than control students. Both economically advantaged and disadvantaged 
students in immersion schools grew in proficiency at faster rates than their control-group counterparts. 
Thus, economically disadvantaged students in Technology Immersion schools reached levels of 
technical proficiency that equaled the proficiencies of advantaged students in control schools. 

Students in Technology Immersion schools used technology applications more often in their 
core-subject classes and they interacted more often with their peers in small-group activities. For 

Cohorts 2 and 3, the yearly growth rates in Classroom Activities for economically advantaged and 
disadvantaged immersion students ranged from 0.19 to 0.43 scale-score points (on a 5-point scale), 
compared to 0.10 to 0.21 points for comparable control-group students. Students in immersion schools 
also had more frequent collaborative learning experiences. Seventh and eighth graders in immersion 
schools reported increasing opportunities for small-group work with classmates, whereas their control- 
group peers reported less frequent small-group activities as they advanced to higher grade levels. 

As laptops aged over four years, students at Technology Immersion schools, compared to 
control, reported more technical problems when they used computers at school. In the fourth 
year, students in Technology Immersion schools reported technical problems with computers at more 
than twice the rates reported by control students. Eighth graders (Cohort 2) and seventh graders 
(Cohort 3) who often inherited second-hand laptops and had used those laptops across school years 
reported significantly more technical problems than control group-students. Although various 
technical problems occurred rarely (a few times a year) or just sometimes (once or twice a month), 
problems with deteriorating laptops substantially increased the workloads of technical-support staff, 
who often were already overburdened with technical demands. 

Across four evaluation years, there was no evidence linking Technology Immersion with student 
self-directed learning or their general satisfaction with schoolwork. Findings from three student 
cohorts across four evaluation years showed there was no statistically significant effect of Technology 
Immersion on student Self-Directed Learning, as measured by the Style of Learning Inventory. As both 
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immersion and control students progressed from lower to higher grade levels, their responses to 
statements measuring self-direction (e.g., goal setting, self-efficacy beliefs, and intrinsic effort) 
revealed significantly negative growth trends. Similarly, there was no significant difference in the 
levels of satisfaction with schoolwork expressed by treatment and control students. Across all middle 
schools, students’ became less satisfied with the meaningfulness and relevance of their schoolwork as 
they advanced to higher grade levels. 

Across four years, students in Technology Immersion schools consistently had fewer disciplinary 
actions than control-group students. Disciplinary Action Reports submitted to the TEA for each 
student during the 2007-08 school year, similar - to the previous three years, showed that immersion 
students had proportionately fewer disciplinary problems than their counterparts in control schools. In 
the fourth year, Cohorts 2 and 3 immersion students had an average of 0.54 and 0.45 disciplinary 
actions per student, respectively, compared to 0.76 and 0.71 per-student averages for control students. 
Effect sizes measuring the magnitude of differences between groups were small (-.11 to -.13). 
However, reducing disciplinary actions may have practically important benefits due to increased 
students who remained in classrooms, and decreased time and effort expended by middle school 
teachers and administrative staff in addressing the disciplinary problems of students removed from 
classrooms. 

For the first-through-third evaluation years, students at Technology Immersion schools had 
significantly lower school attendance rates than control students — however, in the fourth year, 
attendance-rate differences between treatment and control students were smaller and 
statistically nonsignificant. Unexpectedly, students at Technology Immersion schools attended 
school less regularly than control students across the first three years. For example, in the third year, 
economically advantaged and disadvantaged Cohort 2 students had average attendance rates of 96.9% 
and 95.9% compared to 97.2% and 96.3% for their control-group peers. Cohort 2 students at 
immersion schools (who were eighth graders in the fourth year - ) continued to have slightly lower 
attendance rates (96.6% and 95.4% for economically advantaged and disadvantaged students, 
respectively) compared to their control-group counterparts (97.0% and 95.8%, respectively); however, 
differences between groups were extremely small and not statistically significant. Likewise, the 
attendance rate differences between Cohort 3 treatment and control students were very small and 
statistically insignificant in the fourth year. It is possible that the introduction of laptops in some 
control schools during the fourth year may have had a slightly negative effect on students’ school 
attendance rates (similar - to the lower attendance of immersion students). Nevertheless, contrary to 
what might be expected, immersion students’ modestly lower average school attendance rates were not 
associated with lower academic achievement. 

Effects of Technology Immersion on Academic Achievement 

For analyses of student achievement involving comparisons across grade levels, TAKS scale scores 
were standardized as T scores with a mean of 50 and standard deviation of 10. We used three-level 
HLM models to estimate the effects of Technology Immersion on students’ test scores for Cohort 2 
(eighth graders) and Cohort 3 (seventh graders). We also investigated the TAKS performance of 
Cohort 1 students (ninth graders) who attended Technology Immersion and control schools and then 
attended mostly traditional high schools through TAKS testing in spring 2008. The table below 
summarizes the estimated magnitude of the Technology Immersion effect on TAKS reading and 
mathematics achievement across student cohorts. HLM model-based estimations of effects are described 
as the cumulative growth in T-score units for Technology Immersion and control groups, the mean 
cumulative growth differences between groups in T-score units, and the estimated sizes of the effects in 
standard deviation units. Major findings for TAKS achievement follow the table. 
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HLM Model-Based Estimations of Technology Immersion Effects on TAKS scores 
by Subject, Economic Disadvantage Status, and Student Cohort 


Assessment/Student Cohort 

Cumulative Growth 

Standard 

Deviation 

Units 

Immersion 

'/’-score 

Growth 

Control 
T- score 
Growth 

Mean 
T- score 
Difference 

TAKS Reading, Advantaged 

Cohort 1: 9th graders, post-immersion 

0.76 

-0.06 

0.8 1 1 ' 

0.08 

Cohort 2: 8th graders, 3 immersion years 

1.10 

0.39 

0.70 

0.07 

Cohort 3: 7th graders, 2 immersion years 

0.00 

-0.21 

0.21 

0.02 

TAKS Reading, Disadvantaged 

Cohort 1 : 9th graders, post-immersion 

2.22 

1.41 

0.8 1 1 ' 

0.08 

Cohort 2: 8th graders, three immersion years 

2.02 

1.32 

0.70 

0.07 

Cohort 3: 7th graders, two immersion years 

0.22 

0.01 

0.21 

0.02 

TAKS Mathematics, Advantaged 

Cohort 1 : 9th graders, post-immersion 

2.14 

0.88 

1.25 

0.13 

Cohort 2: 8th graders, 3 immersion years 

1.27 

-0.69 

1.96* 

0.20 

Cohort 3: 7th graders, 2 immersion years 

0.74 

-0.83 

1.57* 

0.16 

TAKS Mathematics, Disadvantaged 

Cohort 1 : 9th graders, post-immersion 

1.63 

0.38 

1.25 

0.13 

Cohort 2: 8th graders, three immersion years 

1.81 

-0.15 

1.96* 

0.20 

Cohort 3: 7th graders, two immersion years 

0.09 

-1.48 

1.57* 

0.16 


Note. Estimated T-score growth for students attending schools with average levels of poverty. Cumulative growth in T- 
score units (mean= 50, standard deviation = 10). Standard deviation units = T-score difference/ 10. Cumulative growth for 
Cohorts 1, 2, and 3 accounted for growth across four, three, and two years, respectively. 
f p < .10. *p < .05. 

Technology Immersion had no statistically significant effect on TAKS reading achievement for 
Cohort 2 (eighth graders) or Cohort 3 (seventh graders) — however, for Cohort 1 (ninth 
graders), there was a marginally significant and positive sustaining effect of Technology 
Immersion on students’ TAK reading scores. After controlling for student and school poverty, there 
were no statistically significant effects of immersion on the TAKS reading growth rates for either 
Cohort 2 or Cohort 3. The immersion effects were positive but not by statistically significant margins. 
For Cohort 1 there was a statistically significant and positive sustaining effect of immersion on the 
TAKS reading growth rates of ninth graders who had attended immersion middle schools and then 
moved on to mainly traditional high schools {p < .06). The reading achievement of post-immersion 
students increased by 0.19 T-score point per year (0.76 cumulative growth over four years), whereas 
the achievement of control ninth graders decreased by about 0.01 T-score point per year (-0.06 
cumulative growth). Across Cohorts 1 and 2, economically disadvantaged students grew in reading 
achievement at significantly faster rates than their more affluent peers. For TAKS reading, the sizes of 
immersion effects in standard deviation units (.08, .07, and .02) were very small but increased with 
longer exposure to Technology Immersion and through the post-immersion year in high school. 

Technology Immersion had a statistically significant effect on TAKS mathematics achievement 
for Cohort 2 (eighth graders) and Cohort 3 (seventh graders). For Cohort 1 (ninth graders), the 
sustaining effect of immersion on TAKS mathematics scores was positive but not by a 
statistically significant margin. After controlling for student and school poverty, estimated yearly 
TAKS mathematics growth rates for economically advantaged students in immersion schools (0.42 
and 0.37 T-score points per year for Cohorts 2 and 3, respectively) significantly outpaced their control- 
group counterparts (-0.23 and -0.42 T-score points, respectively). Similarly, estimated yearly TAKS 
growth rates for economically disadvantaged students in immersion schools (0.60 and 0.05 T-score 
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points per year for Cohorts 2 and 3, respectively) were significantly more positive than their control- 
group counterparts (-0.05 and -0.74 T- score points, respectively). There were no statistically 
significant differences between the TAKS mathematics outcomes for Cohort 1 post-immersion and 
control-group ninth graders. For TAKS mathematics, the sizes of immersion effects in standard 
deviation units for Cohort 2 (.20) and Cohort 3 (.16) were small but statistically significant {p < .05). 
The estimated immersion effect for Cohort 1 ninth graders in standard deviation units (0.13) was 
similar to the magnitude of the effect detected at the end of their eighth-grade year. 

Similar to the previous year, students’ use of their laptops for Home Learning — a measure of the 
extent to which a student used a laptop outside of school for homework in the four core-subject 
areas or for learning games — was the strongest implementation predictor of students’ TAKS 
reading and mathematics scores. Given variations in the quality of implementation of Technology 
Immersion across schools, classrooms, and students, two-level HLM models with students nested 
within reading and mathematics teachers were used to examine the relationship between 
implementation and student achievement. Controlling for student characteristics and prior 
achievement, and other variables in the analysis, a composite measure of Student Access and Use 
(Laptop Access Days, Core-Content Learning, and Home Learning) was a consistently positive 
although not always statistically significant predictor of students’ TAKS reading and mathematics 
scores for Cohorts 2 and 3. Of the three elements of Student Access and Use, students’ use of laptops 
for Home Learning was the strongest predictor of both TAKS reading and mathematics achievement. 
For Cohort 2, the extent of Home Learning was a positive but statistically nonsignificant predictor of 
TAKS reading scores and a positive and marginally significant predictor of TAKS mathematics 
scores. For Cohort 3, the extent of Home Learning was a positive and statistically significant predictor 
of both TAKS reading and mathematics scores. 

The findings for Home Learning underscore the important role that individual student laptops play in 
promoting ubiquitous learning and in equalizing the out-of-school learning opportunities for students 
in disadvantaged family and school situations. Individual student laptops, in contrast to laptops on 
carts or computers in libraries, labs, or classrooms, expand where and how student learning occurs. 
However, schools and teachers also played an important role. In a third-year study, researchers found 
that teachers at higher Technology Immersion schools encouraged students’ use of laptops outside of 
school by engaging students in projects or assignments that motivated students to continue working 
outside of class. Also, access to electronic textbooks on laptops motivated many students to continue 
working on chapter assignments outside of school (Shapley et al., 2008). 

Conclusions about the effects of Technology Immersion on TAKS social studies and science 
scores remain in doubt. However, outcomes for TAKS writing, which involved the 
administration of the TAKS assessment in traditional paper-and-pencil format, have 
consistently favored control students although not by statistically significant margins. Since 
TAKS tests for social studies, science, and writing arc not administered annually, immersion effects 
for those subject areas cannot be replicated across cohorts and years. Accordingly, it is difficult to 
draw definitive conclusions about the effects of Technology Immersion for these subject areas. 
Available results have revealed no statistically significant differences between treatment and control 
groups for TAKS social studies, science, or writing scores. Treatment-control group differences for 
science and social studies have varied from year to year, whereas outcomes for TAKS writing have 
consistently favored students at control schools. Across evaluation years, seventh graders in 
immersion schools, on average, have had lower TAKS writing scores (-0.91, -0.28, and -0.73 T-score 
points for Cohorts 1, 2, and 3 students, respectively). Even so, it is possible that the administration of 
the TAKS assessment in paper-and-pencil format may underestimate the writing performance of 
Technology Immersion students who have used word processing software on a regular basis for 
written schoolwork. Some research studies have shown that traditional assessments misjudge the 
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writing performance of students who are accustomed to using word processors for writing and are not 
allowed to use word processors when tested (Russell & Haney, 1997; Russell & Plati, 2001). 


Nature of Fourth-Year Implementation 

The overall level of implementation of the Technology Immersion model increased to some 
extent across years — even so, just a quarter of schools reached substantial levels of immersion 

by the end of the fourth implementation year. Implementation of the Technology Immersion model 
requires Leadership, Teacher Support (buy-in), Parent and Community Support, Technical Support, 
and Professional Development. Given adequate supports, teachers should reach high levels of 
Classroom Immersion, and Student Access and Use of technology should be robust. Mean immersion 
standard scores showed small yearly increases across most implementation support components and 
increases in teachers’ levels of Classroom Immersion. In contrast, the level of Student Access and Use 
declined across years. Mean fourth-year standard scores (ranging from 2.69 to 3.19 on a 4-point 
implementation scale) showed that many schools needed stronger supports, especially in the areas of 
parent and community support for technology use, technical supports that addressed obstacles to 
technology use, and professional development for teachers. 

Core-subject teachers at the majority of schools reported only partial levels of Classroom 
Immersion in the fourth year. Teachers’ mean scores at a fifth of schools, however, revealed 
substantial levels of Classroom Immersion. As a whole, standards-based implementation scores for 
Classroom Immersion increased slightly across years (from 2.48 to 2.69 on a 4-point scale). Scores for 
four of the five elements of Classroom Immersion showed somewhat stronger implementation in the 
fourth year, with the largest increase for teachers’ use of technology for their own puiposes 
(Professional Productivity) and the smallest change for classroom integration (Technology 
Integration). The frequency with which teachers allowed students to use technology for learning 
activities (Student Activities) remained relatively stable across years. 

Students’ access to and use of laptops for learning within and outside of school continued to fall 
well short of expectations in the fourth year. The percentages of schools with at least partial levels 
of Student Access and Use decreased across three years (76%, 68%, and 57%), while the percentages 
of schools with minimal student access and use increased (24%, 32%, and 43%). Several factors 
affected students’ opportunities to use laptops for learning within classrooms and outside of school. 
These factors mainly included time lost for repairs due to aging laptops, schools that opted to transfer 
laptops from individual students to carts or classroom sets, schools that restricted students’ use of 
laptops outside of school, and teachers’ preferences regarding classroom laptop use. Year-to-year 
comparisons showed that the mean implementation level for Laptop Access Days increased between 
the third-and-fourth implementation years (from 2.50 to 2.64 on a 4-point scale) due to more 
consistent student “access” to laptops (although not “ownership”) on carts or as classroom sets at some 
schools. At the same time, the yearly mean implementation levels for Core-Content Learning 
(classroom laptop use) decreased across years (2.07, 2.12, and 1.95) and laptop use for Home 
Learning, likewise, decreased over time (1.75, 1.84, and 1.63). These trends replicate what other 
researchers have documented. When teachers are the “gatekeepers” of technology use, many teachers, 
especially veterans, will opt to continue traditional practices and reject practices that require 
innovation and instructional change (Cuban, 2002; Russell, Bebell, & Higgins, 2004). 
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Implementation and Sustainability 

Implementation Fidelity of the Technology Immersion Model 

During spring 2008 site visits at schools, researchers asked principals, technology specialists, and 
teachers to describe their progress in implementing Technology Immersion, and in retrospect, what 
they would have done differently to improve implementation. 

Nearly all of the Technology Immersion Pilot (TIP) grantees said the lack of a start-up year for 
planning was a major barrier to effective implementation of Technology Immersion. The majority of 
middle schools received their TIP grant awards just before the start of the first project year. Thus, 
many thought implementation would have progressed more smoothly if there had been a start-up year 
to plan for immersion. Respondents said a planning year would have allowed them to (a) have 
conversations with teachers about the decision to become an “immersed school,” (b) develop a plan 
for managing laptops (especially at campuses with larger enrollments), (c) build the school’s 
infrastructure for wireless technology, (d) have teachers become more accustomed to laptops and 
available software and digital resources, (e) provide professional development for teachers to 
strengthen their technical skills and ability to plan technology-integrated lessons, and (f) give teachers 
a chance to “try out” lessons with laptops. 

TIP grantees who were more successful thought that committed leaders, thorough planning, 
teacher buy-in, preliminary professional development for teachers, and a commitment to the 
transformation of student learning were keys to their successfid implementation of Technology 
Immersion. Respondents at some schools attributed effective implementation to several factors. 
Foremost, despite a quick start, district and school administrators had a well-conceived plan for 
implementation, were excited about the project, and listened to teacher input. Administrators had 
“high expectations” for technology use, but they allowed time for teachers to become comfortable. 
Professional development typically began before the first year started and was ongoing across 
implementation years. These schools also had collegial cultures, with teachers saying they learned 
from other teachers, “were all in this together,” and “were willing and ready to try” new practices. 
Improvement of students’ learning experiences drove higher quality implementation. Despite myriad 
laptop management issues, respondents believed challenges had been worthwhile because one-to-one 
student laptops and digital resources had increased the depth of learning across subject areas, exposed 
students to more real-life experiences, and allowed students to demonstrate greater responsibility. 

Many TIP grantees reported that administrative turnover, noncommittal teachers, insufficient 
professional development, inadequate school infrastructures, and laptop management problems 
were impediments to effective implementation of the Technology Immersion model. Respondents at 
many schools cited obstacles that derailed their implementation efforts. At many schools, constant 
principal turnover caused major set-backs each year and undermined teacher buy-in for immersion. 
Many teachers expressed noncommittal attitudes about Technology Immersion at their schools, which 
seemed to stem from four main sources: (a) frustrations caused by the concurrent distribution of 
laptops to teachers and students in the first year, (b) the insufficiency of their preparation to meet 
technical demands and manage technology-integrated lessons, (c) students’ inconsistent access to 
laptops for classroom activities, and (d) uncertainty about their students’ capacity to handle one-to-one 
laptop access (i.e., students were too young or immature, lacked sufficient technical and keyboarding 
skills, had insufficient prior experience with computers, behaved irresponsibly with expensive laptops, 
or wanted to use technology to “play” rather than “learn”). Many teachers wished that professional 
development had been provided earlier, and that the training received had focused on content-specific 
lesson plans. Teachers new to schools often felt unprepared to deal with laptops in classrooms. 
Additionally, respondents at these schools often cited problems with inconsistent wireless Internet 
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services, insufficient technical staff to deal with laptop repairs in a timely manner, and students who 
did not bring their laptops to school or class regularly. 

Sustainability of the Technology Immersion Model 

As part of site visits, administrators, technology specialists, and teachers also commented on 
sustainability of the Technology Immersion model at their schools. 

Sustainability depended on the commitment of district leaders to Technology Immersion and to 
long-range planning for continuation. Principals and technology specialists at many campuses had 
not been directly involved in planning for the sustainability of Technology Immersion beyond the 
fourth year, and in fact, most said that decisions about continuation would rest with district 
administrators. At other campuses, plans were in place to continue Technology Immersion at middle 
schools, and some districts were planning to expand one-to-one computing to high schools or upper 
elementary grades. Respondents who described explicit plans for continuation cited the key roles of 
the superintendent and board of trustees. Sustainability of Technology Immersion depended on 
planning ahead and being prepared for future years, including actions such as having a plan for the 
replacement of worn and outdated laptops, allocating resources to support continuous teacher 
professional development, and allocating resources to provide technical support. 

Sustainability of Technology Immersion depended on the adequacy of funds to support 
continuation. With TIP funds ending, many campuses were uncertain how Technology Immersion 
could be sustained financially. Given limited local and state dollars for technology, most respondents 
hoped to win additional grant awards to continue their one-to-one laptop programs. Many principals 
were optimistic about their chances of securing grants to support continuation but had doubts about 
receiving financial support from their districts or the state. A few districts and schools, however, had 
used local funds to support Technology Immersion and were considering how local funds could 
support continuation. Continuation at one school depended on a local bond issue. If the bond failed, 
there would be no money for laptops because budget shortfalls had caused cuts in administrative and 
teaching positions, so there was little hope of receiving district money for technology. 

One district had invested substantial local funds to sustain immersion in the middle school and to 
expand one-to-one laptop access into the high school. Although the district was relying on local funds, 
the superintendent believed the state should provide more or more flexible financial support. In 
particular, state funding allocations earmarked by lawmakers for specific programs prevented local 
education agencies from combining state and local funds for school-reform initiatives. Additionally, 
several respondents thought the state should provide additional technology funding so that schools did 
not have to depend entirely on local funds. Having sufficient local funds was an acute problem for a 
property-poor district that depended on state funds and grants to purchase technology, but frequently 
did not qualify for grants due to high TAKS scores. One respondent said the continuation of 
Technology Immersion simply “depended on how much money the state legislature makes available to 
schools.” A few administrators believed traditional paper textbooks are outmoded and state funds 
invested in printing and delivering millions of textbooks across the state should be used to fund 
technology. 

For some campuses, the TIP project was just another grant program, and once funding ended, the TIP 
project would disappear - . Some principals said they would “love to continue immersion” but saw no 
way to financially sustain the current model. One principal said Technology Immersion is only 
sustainable in an “ideal world.” Another administrator said, “If the grant is not renewed, it would be 
the end of one-to-one computing.” 
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Sustainability of Technology Immersion was associated with educators’ beliefs about technology’s 
value for addressing the learning styles and needs of students, and educators’ commitment to move 
toward digital school environments. School leaders who wanted to continue one-to-one laptop 
projects often linked their intentions with hopes for student learning. Administrators cited goals that 
involved moving students “away from drill and practice” and toward “creation of products”; preparing 
students for the 21st century; expanding learning outside of school; exposing students to “worldwide 
cultures”; and making learning “more than regurgitating information back on a test.” Other campuses 
were committed to continuation of Technology Immersion because administrators saw the value of 
being “paperless.” This involved purchasing electronic versions of textbooks (on CDs or online), 
conducting student assessments online, providing online college coursework and virtual learning 
opportunities for students, and reducing personnel costs through shared teachers for coursework 
delivered via videoconferencing. Some administrators said they simply could not “imagine being 
without laptops.” “We would be stepping back in time,” said one respondent. 

Some school administrators were committed to continuation of Technology Immersion, but they 
wondered if an incremental approach to implementation might have improved their long-term 
prospects for sustainability. Some principals, especially those at larger schools, were committed to the 
full immersion model, but they thought it might be easier to move gradually toward full 
implementation of Technology Immersion by introducing student laptops gradually, immersing one 
grade at a time. One administrator, however, explained that the ultimate goal should be school-wide 
implementation because everyone “. . .is on the same page. It is a campus initiative, so the 
conversations are not just horizontal, it is vertical as well. That’s the power of it. . .And the electives, it 
is across the board a whole-campus initiative.” 


Several schools that had great difficulty implementing the Technology Immersion model planned to 
abandon one-to-one student laptop access and return to more conventional configurations of 
educational technology. Some schools that experienced severe problems implementing Technology 
Immersion were considering other options for providing student access to technology. For example, 
one-to-one computing would be sustained only at selected grade levels, student access to laptops 
would be restricted to in-school use, laptops would be distributed as classroom sets, or laptops would 
be put on mobile carts for teacher checkout. Some teachers believed classroom sets of laptops instead 
of individual student laptops would minimize laptop “wear and tear” and “ensure that all students have 
a laptop” in class. Decisions to move toward more traditional technology configurations were typically 
intended to prolong the life of laptops. 

Findings from four years suggest that Technology Immersion can be implemented and is 
sustainable if districts and schools are committed to the model — however, other approaches to 
technology use may be appropriate for some districts and schools. Over four years, it became evident 
that Technology Immersion involved more than just buying laptops for students. Technology 
Immersion is a comprehensive model for transforming the school culture, and the nature of teaching 
and learning, and expanding the educational boundaries of the school. This study shows that 
fundamental school change is difficult and requires a long-term commitment at all levels of the school 
system (board members, superintendent, principals, teachers, students, and parents). Given the 
financial and logistical challenges of implementing and sustaining the Technology Immersion model, 
statewide implementation may not be possible. However, those districts and schools that are 
committed to Technology Immersion should have state support for their innovative school-reform 
efforts. At the same time, other districts and schools should receive support for alternative technology 
initiatives that have research-based evidence of effectiveness. 
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1. Introduction 


Technology Immersion arose as a comprehensive model that would counter the gradual way in which 
most Texas schools have introduced technology into the educational process and change the use of 
technology for teaching and learning in Texas classrooms (Texas Education Agency, 2006). The 
Technology Immersion Pilot (TIP), created by the Texas Legislature in 2003, set forth the vision for 
Technology Immersion in public schools. Senate Bill 396 called for the Texas Education Agency (TEA) 
to establish a pilot project to “immerse” schools in technology by providing a wireless mobile computing 
device for each teacher and student, technology-based learning resources, training for teachers to integrate 
technology into the classroom, and support for effective technology use. The TEA has used more than 
$20 million in federal Title II, Part D monies to fund Technology Immersion projects for high-need 
middle schools through a competitive grant process. Concurrently, a research study, partially funded by a 
federal Evaluating State Educational Technology Programs grant, has evaluated whether student 
achievement improved over time as a result of exposure to Technology Immersion. The Texas Center for 
Educational Research (TCER) — a non-profit research organization in Austin — has been the TEA’s 
primary partner for this four-year evaluation that spanned the 2004-05 through 2007-08 school years. 

Theory of Technology Immersion 

The vision for educational technology endorsed by many educators, leaders, and policymakers has shifted 
over time from the use of particular' technology hardware and software products to technology’s 
incorporation into every aspect of the educational environment. Changing views reflect our growing 
understanding of how students learn and how to create environments that enhance teaching and learning. 
Cognitive science and other research reveal that children learn more when they are engaged in 
meaningful, relevant, and intellectually stimulating work (Bransford, Brown, & Cocking, 2003; 

Newmann, Bryk, & Nagoaka, 2001). Many believe that technology can support such learning experiences 
and also enable students to develop competencies needed for the 21st century, such as digital literacy, 
inventive thinking, and effective communication (CEO Forum, 2001; Lempke, Couglin, Thadani, & 
Martin, 2003; Partnership for 21st Century Skills, 2006). 

Similarly, Texas recognizes that the state’s long-term success is tied to the preparation of students for the 
digital age. The Texas Long-Range Plan for Technology, 2006-2020, advances the previous state plan for 
the integration of technology within schools across four domains: teaching and learning; educator 
preparation and development; leadership, administration, and instructional support; and infrastructure for 
technology (TEA, 2006). Senate Bill 396 further defined this comprehensive plan as Technology 
Immersion. Consistent with the overall Texas vision for technology, the long-term aspiration for 
Technology Immersion is to “prepare each student for success and productivity as a lifelong learner, a 
world-class communicator, a competitive and creative knowledge worker, and an engaged and 
contributing member of an emerging global society” (TEA, 2006, p. viii). 

While state statute provided a general description of Technology Immersion, school-based 
implementation of the intervention required additional detail. In specifying the critical components of the 
immersion model, TEA staff considered current research on educational technology as well as practical 
wisdom gained through pilot studies and statewide technology initiatives. Technology Immersion 
assumes that effective technology use in schools and classrooms requires robust technology access, 
technical and pedagogical support for implementation, professional development for educators in using 
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technology effectively, and readily available curricular and assessment resources that support the state’s 
foundation curriculum (English language arts, mathematics, science, and social studies). 

Purpose of the Study 

The overarching purpose of this study was to scientifically investigate the effectiveness of Technology 
Immersion in increasing middle school students’ achievement in core academic subjects as measured by 
the Texas Assessment of Knowledge and Skills (TAKS). The evaluation has also examined the 
relationships among contextual conditions. Technology Immersion, intervening factors (school, teacher, 
and student), and student achievement. The research design is quasi-experimental with 42 middle schools 
assigned to either treatment or control groups, with 21 schools in each. This report combines information 
gathered during the 2007-08 school year with information collected during the previous three school years 
to answer the following evaluation questions: 

• How was Technology Immersion implemented, 

• What was the effect of Technology Immersion on teachers and teaching, 

• What was the effect of Technology Immersion on students and learning, 

• Did Technology Immersion affect student achievement, and 

• What factors were associated with implementation and student outcomes? 

Theoretical Framework for Technology Immersion 

The Theoretical Framework for Technology Immersion has guided the evaluation (see Figure 1.1). The 
experimental design, as illustrated in the framework, allowed researchers to estimate of the effects of 
Technology Immersion, which is the difference between the treatment and control groups. We also 
postulated a linear sequence of causal relationships. Program implementation comes first. Experimental 
schools are to be “immersed” in technology through the introduction of Technology Immersion 
components. The quality of implementation reflects the robustness of wireless laptop access for teachers 
and students, the adequacy of technical and pedagogical support services to maintain an immersed 
campus, the extent to which professional development supports curricular integration of technology, and 
how well curricular resources and assessments are used. 

Given quality implementation, we theorized that an improved school environment for technology would 
then lead to teachers who had greater technology proficiency, had students use technology more and in 
new ways in their classrooms, and used laptops and digital resources to increase the intellectual challenge 
of lessons. In turn, these improved school and classroom conditions would lead students to greater 
technology proficiency, more frequent classroom technology activities, more opportunities for peer 
collaboration, greater personal self-direction, and stronger engagement in school and learning. Student 
mediating valuables presumably contribute to increased academic performance as measured by 
standardized test scores. In the framework, links also are shown between student achievement and 
student, family, and school characteristics, which exert their own influence on learning. The research 
literature underpinning the Technology Immersion model and the theoretical framework is included in 
Appendix A. 
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Figure 1.1. Theoretical Framework for Technology Immersion 
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Organization of the Report 


Data collection in the fourth project year (2007-08) involved a mix of quantitative and qualitative data 
sources. Researchers have annually conducted online teacher surveys and student paper-and-pencil 
surveys. We also have gathered school and student data on a yearly basis from the Texas Public 
Education Information Management System (PEIMS), the Academic Excellence Indicator System 
(AEIS), as well as data on student disciplinary actions from schools. Additionally, researchers have 
visited each of the middle schools in fall 2004 and again in spring 2005, 2006, 2007, and 2008. For this 
report, we include data from observations in a sample of grades 6, 7, and 8 core-subject classrooms. 

Report sections are organized around findings relative to the study’s research questions. An overview of 
report chapters is provided below. 

• Chapter 1, Introduction , provides background on the Technology Immersion project as well as 
the study’s theoretical framework. The chapter also establishes the purpose for the study and the 
research questions addressed. 

• Chapter 2, Methodology, presents information on the evaluation design, characteristics of 
treatment and control schools, study limitations, study participants, data collection methods, and 
data analysis procedures. 

• Chapter 3, Technology Immersion — Fourth-Year Implementation, describes progress toward 
implementation in the fourth year and compares the level of implementation across years. 

• Chapter 4, Effects of Technology Immersion on Teachers and Teaching, presents findings on the 
effects of immersion on teacher variables, including technology knowledge and skills, ideology, 
student classroom activities and peer collaboration, and the intellectual challenge of lessons. 

• Chapter 5, Effects of Technology Immersion on Students and Learning, offers findings on the 
effects of immersion on mediating variables, including students’ experiences with technology; 
their self-perceptions of technology proficiency, self-directed learning, and school satisfaction; 
and their engagement in school and learning. 

• Chapter 6, Effect of Technology Immersion on Studen t Achievement, presents findings on the 
effects of Technology Immersion on academic achievement, as measured by TAKS reading, 
mathematics, writing, science, and social studies. 

• Chapter 7, Factors Associated with Implementation and Outcomes, presents results for 
investigations of the associations between implementation and student academic achievement. 

• Chapter 8, Conclusions and Implications, presents the major findings from the study and 
discusses the implications of outcomes. 
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2. Methodology 


Evaluation Design 

The evaluation design is quasi-experimental. Interested districts and associated middle schools responded 
to a Request for Application (RFA) offered by the Texas Education Agency (TEA) to become 
Technology Immersion Pilot (TIP) schools. Applicants had to meet eligibility requirements for Title II, 
Part D funds (i.e., high-need due to children from families with incomes below the poverty line, schools 
identified for improvement, or schools with substantial need for technology). Twenty-two Technology 
Immersion schools, selected through the competitive grant process, were matched by researchers with 22 
control schools on key characteristics, including size, regional location, demographics, and student 
achievement. Two middle schools from one district (one treatment and one control) were removed from 
analyses in the second year due to damage caused by Hurricane Rita. Thus, fourth-year results are for 21 
treatment and 21 con tied schools. A re-analysis of baseline data for the new comparison groups revealed 
no statistically significant differences between school and student characteristics. Thus, the study’s 
research design remained sound. 

Treatment Sample 

In spring 2004, the TEA released a series of RFAs inviting school districts to apply for TIP grants for up 
to two middle schools. The agency held an external review of proposals, with applications scored and 
rank ordered. Following the external review, researchers and agency staff reviewed proposals to ensure 
that applications met criteria established for Technology Immersion. Final selection of TIP schools 
involved the consideration of several factors, including proposal ratings, size, location, student diversity, 
and academic achievement. Decisions were influenced by the need for geographic distribution and the 
availability of comparable schools for the control group pool. Schools received grants to support the 
implementation of Technology Immersion for four school years. 

Control Sample 

The selection of control campuses first involved the generation of a pool of grades 6 to 8 middle schools 
eligible to receive federal funds for participation in the study. As a next step, researchers identified middle 
schools that matched treatment campuses as nearly as possible on factors, including (a) district and 
campus size, (b) regional location, (c) the proportion of economically disadvantaged and minority 
students, (d) percentage of students passing all TAKS tests, and (e) the gaps between the percentage of 
White students and African American and Hispanic students passing TAKS (all tests). Selection involved 
the use of SPSS ® statistical software procedures to establish parameters around each variable of interest 
and the creation of a computer-generated list of “best matches” for each treatment school. The final 
selection involved a review of the matched list by a team of six researchers to identify the optimal control 
school for each treatment school. Additional schools were selected as alternates in the case that a selected 
control site declined the invitation to participate in the study. This selection process yielded 22 control 
group schools including controls for 8 campuses that came from within the same districts as the treatment 
schools and controls for 14 campuses from closely matched single, middle school districts. 

For the first two evaluation years, each control school received $25,000 annually for study participation, 
with 25% of funds earmarked for professional development as required by Title II, Part D guidelines. At 
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the end of the second year, the TEA offered delayed intervention grants that allowed control schools to 
begin planning for Technology Immersion. Of the 21 control schools included in analyses, 16 (76%) 
applied for and received TIP start-up grants. Grant guidelines allowed control schools to begin planning 
for Technology Immersion in the third year (2006-07); teachers could also receive laptops and 
instructional resources, and schools were required, as in previous years, to use 25% of funds for 
professional development. In the fourth year (2007-08), schools could provide laptops for students. 

Across the 16 control schools, three schools provided laptops for students in grades 6, 7, and 8, six 
schools provided laptops for students in grade 6, four schools provided laptops for students in grade 7, 
two schools provided laptops for students in grade 8, and one school provided laptops for students in 
grades 7 and 8. Control schools that declined Technology Immersion grants continued to receive $25,000 
annually for study participation. 

Characteristics of Participating Schools 

The fourth-year study includes 42 grades 6 to 8 middle schools, including 2 1 treatment and 2 1 control 
schools drawn from rural, suburban, and urban locations in Texas. Middle schools are typically small, 
with more than three-quarters enrolling 600 students or fewer. Schools are highly concentrated in small or 
very small districts (2,999 or less students) across the state, but a third of schools are in large districts 
(10,000 or more students). There are two campus charter schools (one treatment and one control) located 
in a large urban district. 

Results for Mcsts at baseline show that the percentages of economically disadvantaged, minority, English 
as a second language (ESL), and special education students arc statistically equivalent across the 
treatment and control schools (Table 2.1). Likewise, results for student enrollment, mobility, and TAKS 
passing rates show no significant differences. Consequently, the treatment and control schools are 
sufficiently well matched on key demographic and academic performance measures. Additionally, both 
treatment and control groups include a comparable range of campus and district enrollments and schools 
from diverse regions. (See additional statistics in Appendix B.) 

Table 2.1. Comparison of Baseline Characteristics: Technology Immersion (N= 21) and Control 
Schools (N =21) 

[" 95% Confidence Interval for Difference I 


Variable 

Condition 

Mean 

SD 

Lower 

Upper 

t (40) 

Enrollment 

Immersion 

374.9 

348.4 

-284.6 

177.5 

-0.47 


Control 

428.5 

391.3 




Economic disadvantage (%) 

Immersion 

70.8 

17.5 

-3.4 

19.4 

1.42 


Control 

62.8 

19.0 




Minority (%) 

Immersion 

68.1 

28.4 

-10.4 

24.7 

0.83 


Control 

60.9 

27.8 




ESL (%) 

Immersion 

13.5 

17.2 

-1.6 

16.0 

1.66 


Control 

6.3 

9.9 




Special education (%) 

Immersion 

14.7 

5.5 

-4.0 

1.8 

-0.76 


Control 

15.8 

3.7 




Student mobility (%) 

Immersion 

15.8 

4.6 

-3.8 

2.8 

-0.30 


Control 

16.3 

5.9 




TAKS 2004, Passing All (%) 

Immersion 

52.4 

15.7 

-9.2 

8.5 

-0.08 


Control 

52.8 

12.5 




TAKS 2003, Passing All (%) 

Immersion 

65.9 

11.4 

-9.1 

5.5 

-0.50 


Control 

67.6 

12.0 





Source: Texas Education Agency AEIS reports 2004 

Note. TAKS = Texas Assessment of Knowledge and Skills. Differences between groups are statistically 
insignificant. Two campuses (one treatment and one control) were excluded from the groups in the second year. 
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Considering baseline statistics, the sample selection process and matching procedures appear to have 
produced a sample of schools with good internal validity, in that there are no large, statistically significant 
treatment-control differences. Still, the tendency for immersion schools to enroll somewhat higher 
percentages of minority, economically disadvantaged, and limited English proficient students could affect 
outcomes given known links between disadvantaged status and lower achievement (Sirin, 2005). Another 
threat to internal validity was introduced in the third project year - when control schools began to 
implement elements of the treatment. As noted above, control schools began to plan for Technology 
Immersion in the third year, and most of the control teachers received new laptops and instructional 
resources. And, while teachers at control schools had opportunities for technology-related professional 
development during the first two project years, the emphasis intensified in the third and fourth years as 
schools purchased technology-related professional development services from vendors (Dell/Pearson 
Learning Group and Apple). The provision of laptops for students at the control schools introduced 
another Technology Immersion component that could contribute to an underestimation of the magnitude 
of the treatment effect in the fourth year. In particular, records submitted by schools indicated that about 
260 eighth graders (9%) and 480 seventh graders (17%) at control schools received individual laptops 
during the fourth year. 

Another limitation of the study is external validity — the extent to which the results of an experiment can 
be generalized from the specific sample to the general population. Schools eligible to become part of the 
treatment group were limited to those serving large proportions of children from families living in 
poverty 1 and middle schools with grades 6 to 8. Only schools that applied for the grant, and submitted 
applications that met a threshold of quality, were eligible for consideration. Due to these restrictions, the 
treatment group is not representative of the average middle school in Texas. 

A majority of students in the sample are economically disadvantaged, with about 67% of sample students 
qualifying for federal free or reduced-price lunch compared to 51% for middle schools statewide. Sample 
schools include substantially more Hispanic and fewer White and African American students than state 
averages for middle schools. Overall, about 58% of sample students are Hispanic compared to about 37% 
of Texas middle school students. Conversely, the sample includes fewer African American students (7% 
vs. 14%) and White students (36% versus 46%) compared to the state averages. The sample schools also 
differ structurally from Texas middle schools as a whole. Middle schools in Texas, on average, enroll 
more students (667 vs. 402 in sample schools). Sample schools are located either in small or very small 
districts or large districts, whereas state middle schools are distributed across very small or small, mid- 
sized, and large districts. Differences between sample schools and the state almost certainly reflect 
funding restrictions (Title II, Part D) and the amount of available funds per grant. The maximum grant 
amount ($750,000) fell well short of the amount required to support one-to-one technology in larger 
middle schools. 


Participants 


Students 

Three groups or cohorts of students were included in this study, with Cohort 1 followed for four years, 
Cohort 2 for three years, and Cohort 3 for two years (Table 2.2). Cohort 1 (ninth graders) included a total 
of 5,217 students, with 2,469 treatment students enrolled at high schools and 2,748 control students 
enrolled at high schools; Cohort 2 (eighth graders) included 5,436 students, with 2,578 at treatment 
middle schools and 2,858 at control middle schools; and Cohort 3 (seventh graders) included 5,392 
students, with 2,547 students at treatment middle schools and 2,845 at control middle schools. 


1 Federal definition used: 27% of population or more than 2,500 people living below poverty line. 
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Table 2.2. Student Cohorts by School Year and Grade 



Middle School 

High School 

Year 

Grade 6 

Grade 7 

Grade 8 

Grade 9 

2004-05 

Cohort 1 


2005-06 

Cohort 2 

Cohort 1 



2006-07 

Cohort 3 

Cohort 2 

Cohort 1 


2007-08 


Cohort 3 

Cohort 2 

Cohort 1 


Note. Bold text denotes the current evaluation year. In 2007-08, nearly all of Cohort 1 students 
attended traditional high schools. One small high school had one-to-one laptop access. 


Table 2.3 shows that about 70% of ninth graders (Cohort 1), eighth graders (Cohort 2), and seventh 
graders (Cohort 3) are economically disadvantaged. Comparison groups have similar proportions of 
disadvantaged and minority students, and female and male students. The main difference between groups 
is the greater proportion of limited English proficient (LEP) students in treatment schools (about 7 to 12 
percent more). Treatment schools also have slightly higher percentages of economically disadvantaged 
and Hispanic students. 


Table 2.3. Demographic Characteristics of Students: 2007-08 

Enroll- Eco Ethnicity Gender 

ment Disadv. AA Hispanic White LEP Female Male 

Cohort 1 (Post-Immersion) 

Treatment 

N 2,469 1,763 146 1,803 493 442 1,198 1,271 

% 47.3 71.4 5.9 73.0 20.0 17.9 48.5 51.5 

Control 

N 2,748 1,914 200 1,853 678 309 1,328 1,420 

% 52.7 69.7 7.3 67.4 24.7 11.2 48.3 51.7 

Cohort 2 

Treatment 

N 2,578 1,946 132 1,935 485 535 1,282 1,296 

% 47.4 75.5 5.1 75.1 18.8 20.8 49.7 50.3 

Control 

N 2,858 2,038 232 1,948 661 346 1,388 1,470 

% 52.6 71.3 8.1 68.2 23.1 12.1 48.6 51.4 

Cohort 3 

Treatment 

N 2,547 1,954 110 1,934 489 670 1,234 1,313 

% 47.2 76.7 4.3 75.9 19.2 26.3 48.4 51.6 

Control 

N 2,845 2,050 181 1,999 651 406 1,416 1,429 

% 52.8 72.1 6.4 70.3 22.9 14.3 49.8 50.2 

Note. Spring 2008 student database collected from 21 treatment and 21 control schools 

As Table 2.2 shows, Cohort 1 students left middle schools and enrolled as ninth graders in high schools 
for the 2007-08 school year. Overall, Cohort 1 treatment students attended 188 high schools, while 
control students attended 197 high schools. The mean percentages of economically disadvantaged 
students at these schools were similar for treatment and control groups (67.1% and 66.6%, respectively). 
Likewise, similar percentages of treatment and control students attended a high schools the same school 
district (89.9% for treatment students and 91.5% for control students). Specifically, 2,219 treatment 
students attended 58 high schools in the same school districts in 2007-08, whereas 2,515 control students 
attended 62 high schools in the same district. High schools attended by treatment and control students in 
their home districts had similar percentages of economically disadvantaged students (61.3% and 67.6%, 





respectively). A small number of students, however, enrolled in high schools in different school districts. 
There were 250 treatment students who attended 130 high schools in different districts, and 233 control 
students attended 135 high schools in different districts. High schools that treatment and control students 
attended had similar percentages of economically disadvantaged students (61.3% and 54.6%, 
respectively). 

During the 2007-08 school year, 1,367 teachers participated in the study, including 612 at treatment 
campuses and 655 at control campuses (Table 2.4). Teachers in comparison groups arc remarkably similar 
in terms of gender, ethnicity, advanced degrees, and average teaching experience. The decline in the 
number of teachers from the baseline to final year reflects the exclusion of two campuses. 

Table 2.4. Demographic Characteristics of Teachers: Baseline and Final Year 



2004-05 

2007-08 


Treatment 

Control 

Treatment 

Control 


N=22 

N=22 

N=21 

N=21 

Number of teachers 

622 

682 

612 

655 

% Female 

65.4 

68.8 

66.3 

68.4 

% Minority 

42.4 

35.3 

45.9 

42.9 

% African American 

7.8 

7.5 

4.4 

4.6 

% Hispanic 

32.2 

26.3 

39.3 

37.0 

% White 

57.6 

64.7 

54.1 

57.1 

% with no degree 

0.0 

2.0 

0.3 

0.9 

% with advanced degree 

21.7 

22.2 

19.7 

19.2 

Average years experience 

10.9 

11.4 

10.7 

11.5 


Data Collection 

Data collection for the project began in August 2004 and continued through spring 2008. As Table 2.5 
illustrates, researchers conducted site visits at each of the middle schools in fall 2004 and again in spring 
2005 through 2008. Additional measures, administered as pre-tests in fall and post-tests in spring, 
included teacher online surveys and student paper-and-pencil surveys. 


Table 2.5. Time Frame for Data Collection by Year 



2004-05 

2005-06 

2006-07 

2007-08 


Fall 

2004 

Spring 

2005 

Fall 

2005 

Spring 

2006 

Fall 

2006 

Spring 

2007 

Fall 

2007 

Spring 

2008 

Site visits (classroom observations ) 

X 

X 


X 


X 


X 

Teacher Questionnaire ( all teachers) 

X 

X 


X 


X 


X 

Teacher Questionnaire (new teachers) 



X 


X 


X 


Student Questionnaire and SLI (Cohort 1) 

X 

X 


X 


X 



Student Questionnaire and SLI (Cohort 2) 



X 

X 


X 


X 

Student Questionnaire and SLI (Cohort 3) 





X 

X 


X 

Texas Assessment of Academic Skills 
(TAKS) 


X 


X 


X 


X 

Attendance 


X 


X 


X 


X 

Disciplinary actions 


X 


X 


X 


X 


Note. Data collection for 22 treatment and 22 control schools in 2004-05 and 21 treatment and 21 control schools in subsequent 
years. TAKS and attendance data were collected for spring 2003 through 2008. SLI = Style of Learning Inventory. 


9 





We also gathered school and student demographic, attendance, and achievement data from the Texas 
Public Education Information Management System (PEIMS) and Academic Excellence Indicator System 
(AEIS). Across four years, individual middle schools submitted student-level data on disciplinary actions. 

Measures 

Instruments measuring mediating and outcome variables included surveys and student performance 
measures. Survey items and scale scores reliabilities arc provided in Appendix C. 

Teacher Questionnaire 

Immersion and control teachers completed an online technology survey in fall 2004 (September to 
October), and teachers new to the schools completed baseline surveys in fall 2005, 2006, and 2007. All 
teachers working at treatment and control schools completed follow-up surveys in spring (April to May) 
of 2005, 2006, 2007, and 2008. The survey included items related to school technology, teachers’ 
technology proficiency and use, and professional development experiences. In fall 2004, 1,271 teachers 
completed surveys (97% of all teachers, 97% of treatment, and 98% of control). In spring 2005, 1,144 
teachers (88% of all teachers, 87% of treatment, and 88% of control) completed surveys. In spring 2006, 
1,175 teachers completed surveys (93% of all teachers, 92% of treatment, and 95% of control). In spring 
2007, 1,208 teachers completed surveys (94% of all teachers, 94% of treatment, and 93% of control). In 
spring 2008, 1,159 teachers completed surveys (91% of all teachers, 87% of treatment, and 95% of 
control). 

School mediating variables. Teachers responded to 33 items pertaining to their perceptions of school 
technology. They rated their strength of agreement with statements on a 5-point scale ranging from 1 
(. strongly disagree ) to 5 ( strongly agree). Maximum likelihood factor analysis with Varimax rotation 
revealed five distinct factors, including Leadership (12 items), Classroom Technology Integration (4 
items). Technical Support (5 items), Innovative Culture (4 items), and Parent and Community Support (2 
items). Measures of internal consistency (Cronbach’s alpha) for school-level factors ranged from 0.66 to 
0.97. 

Teacher mediating variables. Teacher surveys included measures of mediating variables, with items 
pertaining to teachers’ perceptions of Technology Proficiency (27 items), Professional Productivity (17 
items), Student Classroom Activities (17 items), and Collaboration (1 1 items related to teacher 
interactions with colleagues). Additionally, confirmatory factor analysis of items adapted from the Levels 
of Technology Implementation (LoTi) Questionnaire (Moersch, 2001) showed reasonable fit indices for a 
model having Technology Integration (10 items), Learner-Centered Instruction (4 items), and Resistance 
to Integration (3 items) as factors. Cronbach’s alpha reliability coefficients for scales ranged from 0.70 to 
0.98. 

For Technology Proficiency items, teachers indicated their skill level on a 7-point scale with 1 and 2 
indicating low proficiency ( not true of me now), 3, 4, and 5 indicating moderate proficiency ( somewhat 
true of me now), and 6 and 7 indicating proficiency (very true of me now). Measures of integration — 
Technology Integration, Learner-Centered Instruction, and Resistance to Integration — also involved a 
7-point scale ranging from 1 ( not true of me now) to 7 (very true of me now). For Professional 
Productivity, Student Classroom Activities, and Collaboration, teachers used a 5-point scale to rate the 
frequency of activities or interactions: 1 (never), 2 (rarely— e.g., a few times a year), 3 (sometimes-e.g., 
once or twice a month), 4 (often-e. g., once or twice a week), and 5 (almost daily). 
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Student Surveys 

Students completed a paper-and-pencil questionnaire measuring their technology proficiency and use, and 
the Style of Learning Inventory ( SLI ), a measure of self-directed learning (i.e., self-generated behaviors 
oriented toward the attainment of learning goals). Cohort 2 students completed surveys as sixth graders in 
fall 2005 and spring 2006, as seventh graders in spring 2007, and as eighth graders in spring 2008. 

Cohort 3 students completed surveys as sixth graders in fall 2006 and spring 2007 and as seventh graders 
in spring 2008. Cohort 1 students (ninth graders) who advanced to high schools did not complete surveys 
in the fourth year. 

Technology survey. Survey items measured students’ Technology Proficiency (22 items), Classroom 
Activities (12 items), Technical Problems (6 items), Small-Group Work (6 items), and School 
Satisfaction (6 items). Cronbach’s alpha coefficients ranged from 0.77 to 0.94. As a measure of 
Technology Proficiency, students indicated how well they could use various technology applications on a 
5-point scale: 1 (I can do this not at all or barely), 2 (I can do this with some difficulty), 3 (I can do this 
fairly well), 4 (I can do this very well), and 5 (I can do this extremely well). For measures of Classroom 
Activities, Technical Problems, and Small-Group Work, students used a 5-point scale to rate the 
frequency of activities or interactions: 1 (never), 2 (rarely— e.g., a few times a year), 3 (sometimes— e.g., 
once or twice a month), 4 (often— e. g., once or twice a week), and 5 (almost daily). Students rated school 
satisfaction items on a 5 -point agreement scale ranging from 1 (strongly disagree) to 5 (strongly agree). 

Technology survey response rates for students are summarized in Table 2.6. Response rates were in the 
80% to 91% range from fall 2005 through spring 2008. In each time period, there were only small 
differences in response rates between cohorts and comparison groups. 


Table 2.6. Student Technology Survey Response Rates: 2005-06, 2006-07, and 2007-08 



Fall 3 

Spring 2006 

Spring 2007 

Spring 2008 

N 

% 

N 

% 

N 

% 

N 

% 

Cohort 2 

Treatment 

2,209 

84 

2,379 

89 

2,228 

84 

2,110 

82 

Control 

2,405 

86 

2,452 

87 

2,363 

82 

2,422 

85 

All 

4,614 

85 

4,831 

88 

4,591 

83 

4,532 

83 

Cohort 3 

Treatment 

2,233 

86 

— 

- 

2,220 

85 

2,130 

84 

Control 

2,584 

91 

- 

- 

2,464 

87 

2,454 

86 

All 

4,817 

89 

- 

- 

4,684 

86 

4,584 

85 


“Students completed surveys as sixth graders in fall 2005 and fall 2006. 


Style of Learning Inventory. The SLI is a 48-item survey, developed by the Metiri Group (2004), that is 
based on a model of self-regulated learning (Schunk & Zimmerman, 1998). The items on the SLI are 
categorized into 12 scales and three groupings. The three grouping and related scales are listed below. 

• Forethought is defined as influential processes and beliefs that precede efforts to learn (goal 
setting, strategic planning; self-efficacy beliefs; goal orientation; and intrinsic interest), 

• Performance/Volition control refers to processes that occur during learning efforts and affect 
concentration and performance (attention focusing, self-instruction, imagery; self-monitoring; and 
help seeking), and 

• Self-reflection involves processes that occur after learning efforts and influence a learner’ s 
reaction to that experience. Since the learning process is cyclical, these processes will in turn 
influence forethought regarding subsequent learning efforts (self evaluation, attributions, self 
reactions, and adaptivity). 
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Students rated statements regarding their personal self-direction on a 7-point scale, ranging from 1 
{completely false) to 7 {completely true). Confirmatory factor analysis of fall 2004 SLI data revealed low 
convergent validity of the scales and groupings and no discriminant validity. In addition, the scales and 
groupings were not internally consistent (a = 0.18 to 0.52). Because of these findings, analyses were 
limited to the SLI total score (a = 0.89). 

Given consistent results across three study years showing no significant difference between treatment and 
control students’ self-direction as measured by SLI total scores, we limited the administered of the SLI in 
spring 2008 to Cohort 2 students only. Table 2.7, which summarizes SLI response rates, shows that the 
rates for Cohorts 2 and 3 students have ranged from 77% to 89% across time periods. With the exception 
of the spring 2005 SLI administration, there were only small differences in response rates between cohorts 
or comparison groups. 


Table 2.7. Style of Learning Inventory Response Rates: 2005-06, 2006-07, and 2007-08 



Fall 3 

Spring 2006 

Spring 2007 

Spring 2008 

N 

% 

N 

% 

N 

% 

N 

% 

Cohort 2 

Treatment 

2,115 

80 

2,198 

82 

2,201 

83 

1,989 

77 

Control 

2,265 

81 

2,228 

79 

2,368 

82 

2,318 

81 

All 

4,380 

80 

4,426 

80 

4,569 

83 

4,307 

79 

Cohort 3 

Treatment 

2,173 

84 

- 

- 

2,209 

85 

- 

- 

Control 

2,534 

89 

— 

- 

2,434 

86 

- 

- 

All 

4,707 

87 

- 

- 

4,643 

85 

- 

- 


“"Students completed the Style of Learning Inventory as sixth graders in fall 2005 and 2006. 


Observation of Teaching and Learning 

Researchers have conducted classroom observations for core-subject teachers (reading/English language 
arts, mathematics, science, and social studies) who instructed Cohorts 1, 2, and 3 students. In fall 2004 
and spring 2005, we observed in a sample of sixth-grade classrooms. In spring 2006, we observed a 
sample of classrooms including sixth- and seventh-grade teachers. In spring 2007 and 2008, we observed 
a sample of classrooms including sixth-, seventh-, and eighth-grade teachers. 

The Observation of Teaching and Learning (OTL) form documents basic descriptive information (e.g., 
number of students, content area), technology access and use (i.e., technology available and used by the 
teacher and students), and classroom environment (i.e., organization and management). In addition, 
researchers used time-interval ratings to record information in six areas: class organization (e.g., 
individual students, pairs, small groups, whole group), teacher activities (e.g., directing, guiding 
substantive discussion), teacher’s technology use (e.g., peripherals, presentation software), student 
activities (e.g., listening, learning facts, definitions, algorithms), students’ technology use (e.g., express 
themselves in writing, learn/practice skills), and student engagement (rated on a 5-point scale from low 
engagement to high engagement). 

Observers made the first rating after observing for 5 minutes, then made a rating every 10 minutes. 

During the observation, observers also recorded descriptive notes on the lesson objectives, teachers’ 
questioning strategies (lower or higher order), and class activities. Observations lasted about 45 minutes. 
After the observation, and based on time-interval ratings and descriptive notes, observers rated the 
intellectual challenge of classroom work. Relying on rubrics developed by Newmann, Secada, and 
Wehlage (1995), observers rated four standards measuring the intellectual quality of classroom instruction 
on a 5-point scale: Higher Order Thinking, Disciplined Inquiry, Substantive Conversation, and Value 
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Beyond School. An aggregate score across three of the standards was used as an overall measure of the 
Intellectual Challenge of instruction. We excluded the Substantive Conversation standard because ratings 
were biased by teachers’ classroom organization. Classes with teacher-directed instruction typically 
provided more public conversations, and thus, better opportunities to document the nature of 
conversational exchanges. 

Number of observations. During fall 2004, researchers conducted observations at half of middle schools 
(11 treatment and 1 1 control). Subsequently, we expanded observations to all of the middle schools. In 
fall 2004, researchers observed 125 classrooms (60 treatment and 65 control); in spring 2005, we 
conducted follow-up observations, when possible, in the same classrooms. Altogether, we observed 206 
classrooms (105 treatment and 101 control) in spring 2005. The following year (spring 2006), we 
observed 217 classrooms (114 treatment and 103 control). These observations included a nearly equal 
mix of sixth- and seventh-grade classrooms. In spring 2007 and 2008, respectively, we observed 194 
classrooms (95 treatment and 99 control) and 230 classrooms (117 treatment and 1 13 control). These 
observations included a combination of sixth-, seventh-, and eighth-grade classrooms. At small campuses, 
researchers observed nearly all core-subject teachers. For larger campuses, we observed a representative 
sample of core teachers. 

Training procedures. Prior to site visits in fall 2004 and spring of 2005, 2006, and 2007, researchers 
participated in one- or two-day training events. Training activities informed data collectors about the 
research design, aspects of Technology Immersion, data collection protocols, effective interview and 
focus group techniques, and classroom observation procedures. Approximately half of each training event 
was devoted to the establishment of inter-rater agreement on the OTL form. During observation training, 
raters first reviewed background information and individual item and code definitions in the OTL manual. 
Raters next viewed a video in which a classroom teacher used technology as paid of a lesson. The trainer 
stopped raters at 10-minute intervals to record ratings, discuss the extent of agreement or disagreement, 
and resolve misunderstandings. This process was repeated for an additional classroom video. 
Individualized training was provided for new researchers in 2008. 

To further enhance inter-rater agreement, raters were paired for observations in classrooms during visits 
to a middle school selected for training puiposes. Following paired classroom observations in these 
schools, raters again discussed assigned ratings and resolved disagreements. Subsequently, for site visits 
to treatment and control middle schools, observers were paired for about 25% of classroom observations. 
Overlapping observations allowed the calculation of the consistency of observers’ scores (i.e., the 
percentage of agreement on ratings from paired observations). Additionally, paired observations 
supported the use of Many-Facet Rasch Measurement (MFRM) to adjust scores on the Intellectual 
Challenge factor for differences across raters. 

Inter-rater agreement. Inter-rater agreement on the rating scales for the Intellectual Challenge standards 
(Higher-Order Thinking, Disciplined Inquiry, Substantive Conversation, and Value Beyond School) was 
established by calculating the percentage of time observers agreed on ratings from paired observations. 
Analyses of observations from fall 2004 indicated 78% inter-rater agreement. Agreement reached 98% 
when scale categories were allowed to vary by one scale point (on the 5-point scale). Inter-rater 
agreement declined somewhat in spring 2008. Exact agreement for spring 2005 through 2008 was 63%, 
62%, 62%, and 56%, respectively, and 89%, 93%, 96%, and 92%, respectively, when ratings varied by 
one scale point. 

Reliability of scores. Statistics for inter-rater agreement indicated that raters may have had somewhat 
different standards for assigning scores, so we needed to adjust statistically for the differences in the 
severity of raters. An overall measure of Intellectual Challenge for each teacher was constructed using 
MFRM. The quality of instruction measure is an aggregate score across three standards (Higher Order 
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Thinking, Disciplined Inquiry, and Value Beyond School). The measure is adjusted for the relative 
difficulty of each standard and the relative severity (or leniency) of each observer. MFRM analysis 
produces several fit statistics that can be used to measure each observer’ s intrarater reliability or internal 
consistency. One of these, observer infit, weights each standardized residual by its variance and is more 
sensitive to unexpected patterns of small residuals. A second statistic, observer outfit, is an unweighted 
mean-square residual sensitive to outlying residuals (Linacre, 2004). 

There is no fixed rule for setting upper and lower limits for theses fit statistics. “Misfitting” raters have 
been defined as having either a mean-square infit or outfit statistic greater than 1.5 (Lunz, Wright, & 
Linacre, 1990), or the range has been from 0.5 to 3.0 (Myford & Wolfe, 2000). We define a “misfitting” 
observer as one with either a mean-square infit or outfit statistic less than 0.5 or greater than 1.5. This 
defines “misfit” as less than 50% of the variance in ratings than is modeled (a muted pattern) and more 
than 50% of the variance than is modeled (a noisy pattern). Observation data in fall 2004, and spring 
2005, 2006, 2007, and 2008, respectively, resulted in observer infit values from 0.61 to 1.34, 0.61 to 1.34, 
0.43 to 1.59, 0.58 to 1.14, and 0.62 to 1.59. Observer outfit values over the five time periods ranged from 
0.62 to 1.20, 0.62 to 1.20, 0.40 to 1.67, 0.66 to 1.17, and 0.65 to 1.48. While the spring 2006 and spring 
2008 fit statistics extended slightly beyond the 0.5 to 1.5 range, mean infit and outfit values were in the 
0.90 to 1.00 range. No unusual rating patterns appeared to be present in the spring 2006 and spring 2008 
classroom observation data, with only slightly unpredicted or overly predictable ratings (Linacre, 1995). 

Texas Assessment of Knowledge and Skills (TAKS) 

The TAKS is Texas’ criterion-referenced assessment that annually measures students’ mastery of the 
state’s content standards. TAKS assesses reading at grades 3 to 9; English language aits at grades 10 and 
11; writing at grades 4 and 7; mathematics at grades 3 to 11; science at grades 5, 8, 10, and 11; and social 
studies at grades 8, 10, and 11. Stringent quality control measures are applied at all stages of test 
administration, scanning, scoring, and reporting. Internal consistency reliabilities for TAKS assessments 
are in the high .80s to low .90s range. Evidence also supports the content, construct, and criterion-related 
validity of TAKS assessments. 2 

Table 2.8 shows the TAKS completion schedule for Cohorts 1, 2, and 3 students. Students complete 
TAKS reading and mathematics assessments annually, so all student cohorts have pretest and posttest 
measures. For the present study, Cohort 2 students completed TAKS science in 2005 (5th grade) and 
2008 (8th grade), and TAKS social studies in 2008. Cohort 3 students completed the TAKS writing 
assessment in 2005 (4th grade) and 2008 (7th grade). 

Table 2.8. Texas Assessment of Knowledge and Skills Completion Schedule by Student Cohort 

Texas Assessment of Knowledge and Skills (TAKS) 



F 

leading 

Mathematics 

3 

Yritinj 

X 

? 

Social Studies 

< 

Science 

Year 

Cl 

C2 

C3 

Cl 

C2 

C3 

Cl 

C2 

C3 

Cl 

C2 

C3 

Cl 

C2 

C3 

2003 
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- 

- 

X 

- 

- 

X 

- 

- 

- 

- 

- 

- 

- 

- 

2004 

X 

X 

- 

X 

X 

- 

- 

X 

- 

- 

- 

- 
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- 

- 

2005 

X 1 

X 

X 

X 

X 

X 

- 

- 

rx 

- 

- 

- 

- 

X 

- 

2006 

X 

X 

X 

X 

X 

X 

X 

- 

- 

- 

- 

- 

- 

- 

- 

2007 

X 1 

X 

X 

X 

X 

X 

- 

X 

- 

X 

- 

- 

X 1 

- 

- 

2008 

X 

X 

X 

X 

X 

X 

- 

- 

r x 

- 

X 

- 

- 

X 

- 


Note. Cl = Cohort 1, C2 = Cohort 2, and C3 = Cohort 3. Italic text means the TAKS score was used as a pre-test measure. 


Technical information is available on the Texas Education Agency website at http://www.tea.state.tx.us/student 
assessment/resources/techdig04/index.html. 
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At grades 6, 7, and 8, TAKS reading measures four objectives: understanding of culturally diverse written 
texts, knowledge of literary elements, use of strategies to analyze written texts, and application of critical- 
thinking skills. At grade 9, TAKS reading measures three objectives: understanding of culturally diverse 
written texts, understanding of the effects of literary elements and techniques in diverse texts, and the 
ability to analyze and critically evaluate diverse texts and visual representations. In addition to multiple- 
choice items, ninth graders respond to several open-ended (short-answer) items. 

TAKS mathematics at grades 6, 7, and 8 measures six objectives: numbers, operations, and quantitative 
reasoning; patterns, relationships, and algebraic reasoning; geometry and spatial reasoning; concepts and 
uses of measurement; probability and statistics; and mathematical processes and tools used in problem 
solving. TAKS mathematics at grade 9 measures 10 objectives: functional relationships; properties and 
attributes of functions; linear functions; formulation and use of linear equations and inequalities; 
quadratic and other nonlinear functions; geometric relationships and spatial reasoning; two- and three- 
dimensional shapes; concepts and uses of measurement and similarity; percents, proportional 
relationships, probability, and statistics; and underlying processes and mathematical tools. Each ninth 
grader must have a graphing calculator for use during the test. All TAKS mathematics tests include a 
combination of multiple-choice and open-ended griddable response items. 

At grade 7, TAKS writing measures six objectives: given a context, produce an effective composition for 
a specific puipose; demonstrate a command of conventions of spelling, capitalization, punctuation, 
grammar, usage, and sentence structure; recognize appropriate organization of ideas in written text; 
recognize correct and effective sentence construction in written text; recognize standard usage and 
appropriate word choice in written text; proofread for correct punctuation, capitalization, and spelling in a 
written text. At grade 8, TAKS science measures five objectives: nature of science; living systems and the 
environment; structures and properties of matter; motion, forces, and energy, and earth and space systems. 
Grade 8 TAKS social studies measures five objectives: history, geography, economics and social 
influences, political influences, and social studies skills. 

School Attendance and Disciplinary Actions 

Post-measures of student attendance for Cohort 1 came from PEIMS data for the 2004-05, 2005-06, 
2006-07, and 2007-08 school years; attendance data from 2003-04 served as the pre-measure. Similarly, 
for Cohort 2, student attendance data for 2005-06, 2006-07, and 2007-08 provided post-measures while 
data from 2004-05 served as the pre-measure. Likewise, for Cohort 3, student attendance data for 2006-07 
and 2007-08 provided post-measures and data from 2005-06 served as the pre-measure. Additionally, 
individual campuses submitted data for student disciplinary actions taken during the 2007-08 school year. 
Data files included an indicator for the total number of Disciplinary Action Reports (PEIMS 425 records) 
reported for each student (Cohorts 2 and 3) during the school year. 
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3. Technology Immersion — Fourth-Year Implementation 


Researchers have investigated the implementation of Technology Immersion across four project years. 
Second- and third-year findings showed that many of the 2 1 treatment schools had difficulty 
implementing the prescribed components of the Technology Immersion model. Still, implementation 
varied by campus and some schools reached implementation levels that more nearly met substantial to 
full immersion standards. Given that implementation quality has been associated with desirable project 
outcomes (e.g., Berman & McLaughlin. 1978; Borman, 2005; Borman, Hewes, Overman, & Brown, 

2003; Datnow, Borman, & Stringfield, 2000), we continued to monitor schools’ progress in the fourth 
year. This chapter begins with a description of Technology Immersion and the use of Technology 
Immersion packages as a means to operationally define the treatment and ensure more consistent 
implementation across sites. Next, we describe our approach to measuring implementation. Finally, 
findings are presented on the fidelity of fourth year implementation at the treatment schools, and 
comparisons arc made between the second (2005-06), third (2006-07), and fourth (2007-08) project years. 

Defining Technology Immersion 

As a way to promote consistent interpretation of the Technology Immersion model and comparability of 
implementation across schools, the Texas Education Agency (TEA) issued a Request for Qualifications 
(RFQ) that allowed commercial vendors to apply to become providers of Technology Immersion 
packages (TEA, 2003). State statute provided a general description of Technology Immersion, but the 
concept and its component parts were defined operationally to foster uniformity. Vendors had to include 
six components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus to 
ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as learning tools; 

• Online instructional resources that support the state curriculum in English language aits, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, learning, 
and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through an expert review process, the TEA selected three lead vendors as providers of Technology 
Immersion packages (Dell Computer Inc., Apple Computer Inc., and Region 1 Education Service Center 
[ESC]). Package costs, which ranged from about $1,100 to $1,600 per student, varied according to the 
numbers of students and teachers, the type of laptop computer, and the vendor provider. Of the 21 
immersion sites studied in the second through fourth years, 5 middle schools selected the Apple package, 
15 selected the Dell package, and 1 school selected the Region 1 ESC package (Dell computer). 

Table 3.1 provides an overview of the basic components within each package and the individual vendors 
that provided various products. All vendors offered a wireless laptop as the mobile computing device 
(Apple or Dell), and all laptops had a suite of productivity tools (either AppleWorks or Microsoft Office). 
Dell computers also had a web-based portal ( eChalk ) to applications and resources. 
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Table 3.1. Technology Immersion Packages 


Component 

Apple 

N = 5 Schools 

Dell 

N= 15 Schools 

Region 1 ESC 
N = 1 School 

Wireless laptop computer 

Apple 
iBook G4 

Dell Inspiron 
or Latitude 

Dell 

Inspiron 

Productivity software 

Apple Works 

MS Office 
eChalk 

MS Office 
eChalk 

Online resources 

Various 

Various 

Various 

Online assessment 

AssessmentMaster 

i-Know 

i-Know 

Professional development 

Apple Model 

Pearson Achievement, 
Dell Exchange 

ESC 1 — 

Classroom Connect 

Technical and pedagogical 
support 

Apple, 

Campus/District 

Dell, - 

Campus/District 

ESC 1 — 
Campus/District 


Immersion packages also included a variety of digital resources. Apple provided netTrekker, ClassTools 
Math, ExploreLearning Math and Science, TeenBiz3000 , and My Access Writing. Dell provided 
netTrekker and Connected Tech, and Region 1 ESC provided Connected Tech, Unitedstr earning, 
Encyclopedia Britannica, EBSCO, NewsBank, and K12 Teaching and Learning Center. Packages also 
included formative assessments {AssessmentMaster or i-Know). Additionally, each vendor provided 
professional development as well as ongoing technical support. Apple had its own professional 
development model. Dell relied on a commercial provider ( Pearson Learning Group ) and the Dell 
Exchange (an online resource). Region 1 ESC used a combination of service center support plus services 
offered through Connected Coaching and Connected University. (See Appendix D for a more 
comprehensive description of the package components.) 

During the third and fourth implementation years, schools began to selectively purchase online resources 
and assessments according to their perceived needs. For example, some schools dropped the online 
assessments because they had state -provided or local assessments that filled their testing needs. Two 
schools (with Dell and ESC lpackages) purchased the My Access Writing program included in the Apple 
package. Schools and teachers also continued to supplement package resources with products purchased 
locally, provided through state textbook adoptions, or obtained from the Internet free of charge. 

Measuring Implementation Fidelity 

Implementation is measured as the fidelity with which Technology Immersion components and related 
elements attain an envisioned “ideal.” This approach involved gathering extensive data on immersion 
components at each of the treatment campuses and comparing campus-to-campus variations with the 
vision for “full” implementation. The seven immersion components include five supports for 
implementation (Leadership, Teacher Support, Parent and Community Support, Technical Support, and 
Professional Development) and two components related to teacher and student implementation outcomes 
(Classroom Immersion and Student Access and Use). Consistent with previous years, we used a two-part 
measurement approach in the fourth year. First, we used indicators to describe each school’ s progress on a 
4-stage scale toward immersion standards. Rating scales for components and related elements identified 
four levels of immersion: minimal (0 to 1.99), partial (2.00 to 2.99), substantial (3.00 to 3.49), and full 
(3.50 to 4.00). Second, we used quantitative implementation indices to gauge the level of Technology 
Immersion using standardized scores (z scores). Z scores allowed the calculation of composite scores 
across indicators with varying scales and standard deviations. 
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Implementation Indicators 


Both the immersion standard scores and implementation indices were derived from values for the seven 
components and their related elements. Fourth-year scores came from spring 2008 surveys of teachers 
(N = 534, including 337 core-subject teachers) and students (N = 6,327) at treatment schools. Table 3.2 
provides descriptions of the Technology Immersion indicators. Appendix D provides additional technical 
detail on the measurement of implementation fidelity and the scoring rubrics that described the four levels 
of immersion. 

Table 3.2. Description of Implementation Indicators for Technology Immersion 

Support for Technology Immersion 
Leadership 

To what extent do teachers indicate that administrators establish a clear vision and expectations, encourage integration, 
provide supports, and involve staff in making decisions about instructional technology. 

Teacher Support 

To what extent do teachers share an understanding about technology use, do teachers continually learn and seek new 
ideas, are teachers unafraid to learn about and use technologies, and are teachers supportive of integration efforts. 

Parent and Community Support 

To what extent do teachers believe that parents and the surrounding community support the school’s efforts with 
technology. 

Technical Support 

To what extent do teachers indicate that technical problems with computers, Internet access, repairs, and material 
availability pose barriers to Technology Immersion. 

Professional Development 

Contact Hours: To what extent does the duration (hours) of technology-related professional development (PD) support 
the integration of technology into teaching, learning, and the curriculum. 

Classroom Support: To what extent do core-subject teachers receive coaching or mentoring from an internal source, 
such as another teacher or technology coordinator, or an external (non-school) source. 

Content Focus: To what extent do core-subject teachers indicate that PD emphasizes curriculum, instructional 
methods, and lesson development in core subjects. 

Coherence: To what extent do core-subject teachers indicate that PD is consistent with personal and school goals, 
builds on prior learning, and supports state standards and assessments. 

Classroom Immersion 

Technology Integration: To what extent do core teachers alter instructional practices, allocate time, integrate research 
on teaching and learning, improve basic skills, and support higher order thinking through technology. 

Learner- Centered Instruction: To what extent do teachers have students establish learning goals, use information and 
inquiry skills, complete alternative assessments, and have active and relevant learning experiences. 

Student Classroom Activities: To what extent do teachers have students use particular technology resources for 
learning in core-subject classes, such as a word processor for writing, a spreadsheet for calculation or graphing, or the 
Internet for research. 

Communication: To what extent do teachers use technology to communicate with students, parents, and colleagues or 
to post information on a class website. 

Professional Productivity: To what extent do teachers use technology to enhance their professional productivity (e.g., 
keep records, analyze data, develop lessons, deliver information). 

Student Access and Use 

Laptop Access: To what extent do students have access to wireless Japtops throughout the school year. 

Core-Subject Learning: How frequently do students use technology resources for learning in core-subject classes. 
Home Learning: To what extent do students have access to and use laptops outside of the school for homework and 

learning. 

Note. See Appendix D for a technical description of the measurement of implementation indicators. 
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Computing Implementation Scores 
Scores for Immersion Standards 

We used teacher and student survey data to compute implementation scores for indicators that measured 
progress toward immersion standards (i.e., minimal to full implementation). Adapting a process 
developed by the RAND Corporation, 1 the value for each indicator was computed relative to the 
maximum value (4.00 — the value assigned to full implementation). Standardization based on the 
maximum value allowed comparisons across different types of indicators. For each component and 
element of Technology Immersion, standardization involved the following computations: 

• Agreement scales (i.e., strongly agree or strongly disagree with a prescribed practice or 
behavior): 4 = strongly agree, 3 = agree, 2 = neither agree nor disagree, 1 = disagree, and 
0 = strongly disagree. 

• Frequency scales (i.e., four- or five-level frequencies of doing a prescribed practice): 

4 = highest frequency met, 3 or 2.67 = second highest frequency, 2 or 1.33 = third-highest 
frequency, 1 = fourth-highest frequency, and 0 = never or do not do. 

• Continuous variables (i.e., how much time or how often a prescribed practice is done): 

4 = meet or exceed requirements, and 0-3.99 = proportional fraction of requirement. 

Scores for Implementation Indices 

In addition to the standards-based scoring system described above, we used teacher and student survey 
data to compute standardized implementation indicators (z scores with a mean of 0 and standard deviation 
of 1.0) that could then be aggregated to generate: 

• A single implementation score for each Technology Immersion component for each school 
(e.g.. Leadership Index), 

• a mean implementation support score for the five support components (Support Index), 
including Leadership, Teacher Support, Parent and Community Support, Technical Support, 
and Professional Development, and 

• an overall mean implementation score for each school (Implementation Index), which is an 
average of the Support Index, Classroom Immersion Index, and Student Access and Use 
Index. 2 


Implementation of Technology Immersion 

The sections to follow present findings on (a) the extent to which schools provided the implementation 
supports considered essential to advance Technology Immersion, and (b) the degree to which schools 
implemented components relevant to teachers’ classroom immersion practices and students’ technology 
access and use. We first present results for implementation standards (measured at four levels) that 
describe the extent to which the model’s support components and instructional and learning components 
were implemented as designed. These scores showed whether middle schools attained the standards that 
represented what a substantially or fully immersed campus should achieve. Next, we use implementation 


1 Vernez, G., Karam, R., Mariano, L.T., & DeMartini, C. (2006). Evaluating Comprehensive School Reform Models at Scale: 
Focus on Implementation. Santa Monica, CA: RAND. 

2 

“ Variables were standardized as z scores from their original scale or continuous variable values. The use of z scores rather than 
the immersion standard scores was necessary in order to aggregate data across variables that had widely varying standard 
deviations. 
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indices (z scores) to provide an overall measure of Technology Immersion (Implementation Index) and to 
compare the relative level of implementation for components across schools. 

Implementation Standards 

As explained previously, progress toward Technology Immersion standards was measured at four levels 
( minimal , 0-1.99; partial, 2.00-2.99; substantial, 3.00-3.49; and full immersion, 3.50-4.00) across seven 
components. Five components assessed the strength of supports for Technology Immersion (Leadership. 
Teacher Support, Parent/Community Support, Technical Support, Professional Development), whereas 
one component gauged the extent of teachers’ Classroom Immersion and another component measured 
Student Access and Use (of technology). Figure 3.1 displays the mean implementation scores by 
component and project year. 



Figure 3.1. Mean level of implementation (measured on a 0 to 4 scale) for seven Technology 
Immersion components (N=21 middle schools) by year. 3 

Mean standard scores for Technology Immersion components generally showed small increases across 
years, with the exception of Technical Support (which remained fairly stable: 2.73, 2.82, and 2.76) and 
Student Access and Use (which declined each year: 2.17, 2.15, and 2.07). Fourth-year mean 
implementation support scores ranging from 2.69 (Professional Development) to 3.19 (Teacher Support) 
showed that supports for immersion from school administrators, teachers, the community, technical staff, 
and professional development providers did not reach full implementation standards (mean score of 3.50 
to 4.00). Consistent with the second and third years, teachers, on average, reported only partial levels of 
Classroom Immersion in the fourth year (M = 2.67), and students, as a whole, reported partial levels of 
technology access and use (M = 2.07). Results for individual components, which are discussed in detail 
below, showed that the level of implementation varied considerably across schools. 


3 Standards-based scores for Professional Development, Classroom Immersion, and Student Access and Use are averages across 
elements of these components. These scores serve descriptive purposes. Composite z scores are used in statistical analyses. 
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Level of Principal, Teacher, and Parent/Community Support 


The Technology Immersion model calls for the systemic integration of technology into all aspects of the 
school. Momentum for implementation, thus, depends upon the backing and support of individuals, 
establishment of institutional norms, and assistance from the surrounding community. Sections to follow 
describe teachers’ reported support from key constituents. 

Leadership. Administrators play key roles in setting the direction for Technology Immersion, providing 
resources, and building the capacity of staff. Thus, teachers at each school have been asked every year to 
rate the quality of administrative leadership. Administrators demonstrated leadership through behaviors 
such as involving staff in decisions, setting clear expectations for technology use, encouraging and 
participating in professional development events, and providing resources and support. Results in 
Figure 3.2 show that administrative leadership was relatively stable across three implementation years. 
Teachers at about half of campuses reported substantial levels of leadership. Mean scores across years 
(3.19, 3.25, and 3.17, respectively) indicated that these teachers either agreed or strongly agreed that 
administrators provided technology-related leadership. Teachers in an additional half of schools reported 
partial levels of administrative support (M = 2.64, 2.69, and 2.77, respectively). 



□ Minimal oPartial □ Substantial BFull 


Figure 3.2. Level of implementation (measured on a 0 to 4 scale) for Leadership, by the mean 
implementation score, percentage of schools at each implementation level, and year. 

Teacher Support. Teacher “buy-in” for Technology Immersion is critically important because students’ 
school experiences with technology are largely dictated by their teachers. Thus, it is noteworthy that 
teachers reported increased levels of support for technology innovation across years (Figure 3.3). In the 
fourth year, teachers at two campuses (10%) reported a full level of support (M = 3.78). That is, teachers 
at these schools strongly agreed that they shared an understanding about technology use for student 
learning, were continually learning and seeking new ideas, were not afraid to learn about and use new 
technologies, and were supportive of integration efforts. Teachers at two-thirds of schools reported a 
substantial level of support for technology innovation (M = 3.22). In contrast, teachers at a quarter of 
campuses reported just partial levels of support (M= 2.86). 
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Teacher Support, Year 2 Teacher Support, Year 3 Teacher Support, Year 4 
I □ Minimal □ Partial [^Substantial BFull 


Figure 3.3. Level of implementation (measured on a 0 to 4 scale) for Teacher Support, by the mean 
implementation score, percentage of schools at each implementation level, and year. 

Parent and Community Support. Since parents must share responsibility for an expensive laptop 
computer with their child or children, their understanding of and support for Technology Immersion is 
imperative. Additionally, the enthusiastic support of community members, including elected members of 
the local school board and business people, may influence implementation through mechanisms such as 
the adoption of supportive policies, provision of resources, or promotion of positive public relations. 
Given the importance of parent and community support, teachers’ perceptions of such support are 
important (Figure 3.4). In the fourth year, teachers at less than a third of schools reported substantial to 
full levels of parent and community support (M = 3.16 and 3.63, respectively), with teachers generally 
agreeing that parents and the surrounding community supported their efforts with technology. Conversely, 
teachers at nearly three-quarters of schools reported just partial levels of parent and community support 
(M= 2.65). Fourth-year results represented a reduction in parent/community support compared to the 
third year when teachers at more than a third of schools reported substantial levels of support and less 
than two-thirds of schools had partial support. Thus, garnering parent and community support was a 
greater problem at some schools in the fourth year. 


3.63 



Parent/Community Support, Parent/Community Support, Parent/Community Support, 
Year 2 Year 3 Year 4 


□ Minimal □ Partial □ Substantial BFull 


Figure 3.4. Level of implementation (measured on a 0 to 4 scale) for Parent and Community Support, 
by the mean implementation score, percentage of schools at each implementation level, and year. 
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Level of Technical and Pedagogical Support 

Technical and pedagogical supports are critical aspects of the Technology Immersion model. As schools 
build their network infrastructure and acquire computer hardware and technology resources, ongoing 
technical support for all components of immersion and ongoing professional development in integrating 
technology into teaching and learning are essential for successful implementation. 

Technical Support. Technical support for immersion should be provided by vendor technicians as well 
as district and campus staff who assist with implementation and offer timely support when technical 
problems arise. Results in Figure 3.5 show that the level of technical support improved at some schools 
over time. Teachers at about a third of schools reported substantial or full levels of technical support in 
the fourth year (M = 3.10 and 3.50, respectively). Although teachers at two-thirds of schools reported just 
partial levels of technical support (M = 2.56), this was fewer schools than in the previous two years. 
Teachers at schools with partial implementation were generally unsure that school computers were kept in 
working order, requests for assistance were addressed in a timely way, Internet connections worked 
adequately, and classroom materials were readily available. Despite improvements, technical problems 
continued to challenge teachers at many schools in the fourth year. 


3.50 3.50 

3.12 5% 3.17 3.1Q 5% 



Technical Support, Year 2 Technical Support, Year 3 Technical Support, Year 4 


□ Minimal □Partial □ Substantial ■Full 


Figure 3.5. Level of implementation (measured on a 0 to 4 scale) for Technical Support, by the mean 
implementation score, percentage of schools at each implementation level, and year. 

Professional Development. Each of the Technology Immersion packages included a professional 
development component designed to support all educators on an implementing campus. The immersion 
model required professional development that instructed teachers in effective classroom integration and 
was delivered through proven methods (i.e., learning through a variety of delivery systems, collaboration, 
sustained learning opportunities, and ongoing coaching and support). In addition to professional 
development provided by immersion package vendors, each school was offered a grant in the fourth year 
to participate in the Intel Teach Program. Grant funds paid expenses to train Intel Master Teachers (MTs) 
and provided stipends for at least 10 participant teachers to be trained by the MT. Master Teachers 
participating in the train-the-trainer model selected one of two Intel Teach professional development 
options: Essential Skills Course (development of a curricular unit integrating technology) and Teaching 
Thinking with Technology Course (use of technology tools to advance students’ higher order thinking 
skills). Of the 21 treatment schools, 17 schools received grants to train at least one MT who provided 
school-based training for their peers. 

Although professional development should support all teachers at a school, our implementation measure 
concentrated on core-subject teachers because of their close association with measured student academic 
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outcomes. Year-to-year comparisons displayed in Figure 3.6 for the composite Professional Development 
indicator (mean score for four standards-based elements) show there was little difference in the levels of 
implementation between the second and third project years, but the quality of professional development 
improved at several schools in the fourth year. Although about two-thirds of campuses had minimal to 
partial levels of implementation for professional development in the fourth year (M - 1.81 and 2.50, 
respectively), the remaining campuses achieved substantial or full levels of professional development 
(M= 3.13 and 3.54, respectively). 


3.54 

3 - 20 3.09 3.13 5% 



Professional Development, Professional Development, Professional Development, 
Year 2 Year 3 Year 4 


□ Minimal □ Partial □ Substantial ■Full 


Figure 3.6. Level of implementation (measured on a 0 to 4 scale) for Professional Development, by 
the mean implementation score, percentage of schools at each implementation level, and year 

Figure 3.7 compares the implementation levels for each of the elements that contributed to the composite 
Professional Development measures. Mean immersion standard scores increased in the fourth year across 
all of the professional development indicators, with the greatest improvement for Contact Flours and 
Content Focus. 



□ Year 2 □ Year 3 ■ Year 4 


Figure 3.7. Level of Implementation (measured on a 0 to 4 scale) for elements of the Professional 
Development component by mean implementation score and year. 

Despite annual increases in Contact Flours, core teachers reported receiving less than the prescribed 
number of hours of technology-related professional development in the fourth year (estimated to be about 
50 or more hours per year). The mean implementation score (2.79) indicated that teachers, on average, 
participated in 37 hours or less of technology-related professional development. Additionally, si mi lar to 
previous years, teachers reported that they received just partial levels of classroom support for 
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Technology Immersion (M = 2.26), indicating that teachers as a whole rarely (a few times a year) or 
never received classroom coaching or mentoring from an internal source (such as another teacher or 
technology coordinator) or external source (such as a vendor-provided professional trainer). 

Moreover, teachers as a whole often failed to see the coherence of technology-related professional 
development with their personal goals, earlier learning experiences, and state/district curriculum 
standards and assessments. Like previous years, teachers’ mean rating in the fourth year (2.63) indicated 
that professional development was coherent to a minimal extent (partial implementation). Core-subject 
teachers, however, expressed stronger beliefs in the fourth year about the extent to which professional 
development activities supported their curricular and instructional goals. Teachers mean score of 3.09 
(substantial implementation) indicated that the content of professional development placed a minor to 
major emphasis on curriculum, instructional methods, and lesson development in core areas. 

Level of Classroom Immersion 

Given the needed equipment, digital resources, and support for Technology Immersion, teachers are 
expected to design technology-enhanced learning environments and integrate technology into teaching, 
learning, and the curriculum. Cross-year comparisons for teachers’ composite level of Classroom 
Immersion show that teachers at several schools made progress in creating technology-immersed 
classrooms (Figure 3.8). Teachers at about a fifth of schools had substantial levels of classroom 
immersion in the fourth year (M =3.11), which was twice as many schools compared to the previous year. 
Nevertheless, teachers at a majority of schools reported only partial levels of Classroom Immersion each 
year, although mean scores for partial implementers increased across years (2.45, 2.47, and 2.60, 
respectively). Each year, one school (5%) had a minimal level of Classroom Immersion. 



□ Minimal □ Partial □ Substantial ■ Full 


Figure 3.8. Level of implementation (measured on a 0 to 4 scale) for the Classroom Immersion, by 
the mean implementation score, percentage of schools at each implementation level, and year. 

Figure 3.9 illustrates teachers’ level of implementation relative to five elements of Classroom Immersion: 
Technology Integration, Learner-Centered Instruction, Student Classroom Activities (with technology), 
Communication, and Professional Productivity. On average, teachers reported partial levels of 
implementation across years for four of the five elements of Classroom Immersion. Teachers’ use of 
technology for their own Professional Productivity reached a substantial level of implementation in the 
fourth year (M = 3.04). For most of the elements, except Technology Integration, teachers reported 
slightly stronger implementation in the fourth year, with the largest increase for teachers’ use of 
technology to enhance their Professional Productivity. Comparisons across years indicate that teachers, on 
average, became somewhat more positive about technology integration, learner-centered instructional 
methods, the use of technology as a communication tool, and technology use for professional 
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productivity. In contrast, the frequency with which core-subject teachers had students in their classrooms 
use technology for learning activities remained relatively stable across years. In general, teachers at many 
schools seemed to view technology as a more valuable tool for themselves than for their students. 



Technology Learner- Student Communication Professional 

Integration Centered Activities Productivity 

Instruction 


□ Year 2 □ Year 3 ■ Year 4 


Figure 3.9. Level of Implementation (measured on a 0 to 4 scale) for five elements of Classroom 
Immersion by mean implementation score and year. 

Level of Student Technology Access and Use 

The transformation of classroom experiences is a vital paid of Technology Immersion, but the model also 
aims for students to have on-demand technology access both within and outside of school that allows 
them to become more independent and self-determined learners. Overall, data reported by students 
indicated that Student Access and Use remained relatively stable across the second and third project years 
but declined substantially at several schools in the fourth year (Figure 3.10). 



Student Access and Use, Student Access and Use, Student Access and Use, 
Year 2 Year 3 Year 4 


□ Minimal □Partial ^Substantial ■Full 


Figure 3.10. Level of implementation (measured on a 0 to 4 scale) for the Student Access and Use, 
by the mean implementation score, percentage of schools at each implementation level, and year. 


The percentage of schools with partial levels of Student Access and Use decreased across years (from 
76% to 57%), whereas the percentage of schools with minimal access and use increased (from 24% to 
43%). Moreover, the mean implementation level at schools with minimal access and use declined to the 
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lowest level in the fourth year (1.28 compared to 1.60 and 1.74 in the previous two years). In contrast, the 
mean level of implementation at schools with partial Student Access and Use remained fairly stable 
across project years (2.35, 2.35, and 2.33, respectively). 

Figure 3.1 1 shows the average level of implementation for three elements of Student Access and Use: 
Laptop Access Days, Core-Content Learning, and Home Learning. First, in a fully immersed school, all 
students should have access to their wireless laptops and resources nearly the entire school year (about 
170 to 180 days). Schools as a whole, however, had difficulty keeping laptops in the hands of students. 
Year-to-year comparisons indicated that the mean implementation level for Laptop Access Days declined 
between the second and third years (from 2.69 to 2.50) but improved in the fourth year (2.64). Thus, 
students, on average, had laptops available for a larger number of days in the fourth year. Even so, partial 
levels of implementation indicated that students’ access to laptops varied at schools to a large extent 
(from 100 to 176 days per student). In the fourth year, students at 33% of schools reported either 
substantial or full laptop access. In contrast, students at 57% of schools reported partial access, and 
students at 10% of schools reported minimal laptop access. 

4 n 



Laptop Access Days Core-Content Learning Home Learning 


□ Year 2 □ Year 3 ■ Year 4 


Figure 3.1 1 . Level of Implementation (measured on a 0 to 4 scale) for three elements of Student 
Access and Use by mean implementation score and year. 

Students also estimated how often they used laptops in their English/language aits, mathematics, science, 
and social studies classes and for learning at home. In contrast to improvement in Laptop Access Days, 
there were notable decreases in the fourth year for both Core-Content Learning and Florne Learning. 
Students as a whole reported a minimal level of implementation in the fourth year for Core-Content 
Learning (M = 1.95), suggesting that they rarely (a few times a year) or never used laptops in core-subject 
classrooms. Students, on average, used their laptops even less frequently for learning outside of school in 
the fourth year. Students reported a minimal level of laptop use for home learning each year, and the 
mean level of laptop use for schoolwork outside of school declined substantially in the fourth year (1.75, 
1.84, and 1.63). Thus students, on average, used their laptops outside of school for homework and 
learning either not at all or to a trivial extent. 

Overall, students’ opportunities to use their laptops both within classrooms and outside of school were 
affected by the number of days that students actually had their laptops. In some schools, students’ laptop 
access days were drastically reduced by factors such as time for repairs, technical issues, disciplinary 
infractions, and parent resistance. Students in other schools, contrary to the tenets of Technology 
Immersion, were not allowed to take their laptops home, or their home use was restricted in some way 
(e.g., laptops could only be used for special assignments). A few treatment schools in the fourth year took 
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individual laptops from students and placed them on laptop carts, distributed them as classroom sets, or 
put them in computer labs. Although laptops in such configurations may be available for student use, 
fourth-year findings indicate that deviation from the one-to-one student laptop access model is associated 
with reduced use of laptops for learning in core classes and at home. 

In sum, overall results for the implementation of Technology Immersion as measured by standards-based 
scores show that the levels of support for implementation increased to some extent between the second 
and fourth project years. Similarly, teachers’ reported a slightly increased level of Classroom Immersion, 
although it largely reflected their growing use of technology for their own productivity. In contrast, the 
level of Student Access and Use declined in the fourth year. Findings for standards-based scores also 
showed that the level of implementation varied by campus. By the end of the fourth year, none of the 
middle schools achieved full immersion, and evidence suggested that just a few campuses reached 
substantial immersion, and a majority of schools achieved minimal to partial immersion. 

Implementation Indices 

To further illustrate each school’s level of immersion in the fourth year. Table 3.3 presents the composite 
campus Implementation Index (z score) alongside implementation indices (z scores) for each of the seven 
components. Z scores have a mean of 0 and a standard deviation of 1.0. Thus, the campus score indicates 
how many standard deviations from the mean a score lies. Schools with scores above 0 have higher values 
on the components of Technology Immersion, whereas schools with index values below 0 show less 
evidence of immersion. The Implementation Index is an average score for the Support Index, Classroom 
Immersion Index, and Student Access and Use Index. 

Table 3.3. Fourth-Year Implementation of Technology Immersion 

j Support Index i 


Middle 

School 

(MS) 

Leader- 

ship 

Index 

Teacher 

Support 

Index 

Parent/ 

Comm. 

Index 

Technical 

Support 

Index 

PD 

Index 

Classroom 

Immersion 

Index 

Student 

Access/Use 

Index 

Implemeii' 

tation 

Index 

MS 1 

1.69 

2.19 

2.25 

2.05 

0.88 

1.23 

2.02 

2.58 

MS 2 

1.00 

0.36 

0.15 

0.48 

0.54 

1.52 

1.77 

1.78 

MS 3 

1.30 

0.54 

1.60 

1.11 

-0.28 

0.75 

0.49 

1.08 

MS 4 

0.45 

2.27 

0.73 

-0.16 

2.08 

1.71 

-0.99 

0.99 

MS 5 

0.42 

0.50 

0.38 

-0.83 

0.75 

0.10 

0.47 

0.40 

MS 6 

0.01 

0.36 

-1.06 

0.19 

0.52 

-0.13 

0.97 

0.39 

MS 7 

0.49 

-0.56 

-0.09 

0.98 

0.41 

-0.05 

0.25 

0.25 

MS 8 

0.50 

0.13 

0.70 

0.72 

0.01 

-0.45 

0.20 

0.15 

MS 9 

-0.19 

0.35 

-0.11 

-0.74 

0.94 

-0.19 

0.39 

0.12 

MS 10 

-0.53 

0.03 

-0.57 

-1.48 

0.80 

0.45 

-0.33 

-0.17 

MS 11 

-0.83 

0.94 

-1.58 

-1.67 

-1.30 

1.57 

-0.81 

-0.22 

MS 12 

0.81 

-0.23 

-0.88 

-0.17 

-0.71 

0.25 

-0.42 

-0.23 

MS 13 

-0.02 

-0.81 

0.85 

0.95 

-0.02 

0.66 

-1.50 

-0.25 

MS 14 

0.39 

-0.23 

0.43 

-0.02 

0.09 

-0.35 

-0.41 

-0.25 

MS 15 

-1.29 

-0.23 

0.39 

1.08 

0.96 

-1.31 

0.20 

-0.38 

MS 16 

0.85 

-0.10 

-0.50 

-0.13 

-1.77 

-0.77 

0.14 

-0.49 

MS 17 

-1.61 

-1.58 

-0.12 

-0.12 

-1.05 

-1.21 

0.79 

-0.72 

MS 18 

-2.33 

-0.28 

0.81 

0.79 

-2.05 

-2.07 

0.73 

-0.98 

MS 19 

-0.51 

-1.38 

-1.34 

-1.08 

-0.20 

-0.44 

-1.08 

-1.24 

MS 20 

-0.98 

-1.19 

-0.96 

-1.61 

0.04 

-0.45 

-1.21 

-1.33 

MS 21 

0.36 

-1.10 

-1.32 

-0.34 

-0.61 

-0.82 

-1.69 

-1.49 


Note. Implementation indices are z scores with a mean of 0 and a standard deviation of 1 .0. Scores above zero indicate a 
greater presence of Technology Immersion components and higher levels of implementation. 
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Despite some variations in component scores, middle schools with positive values on the Implementation 
Index tended to have component scores that indicated a stronger presence of the immersion attributes 
such as administrative leadership and teacher support for immersion. In contrast, middle schools that had 
the most negative values on the Implementation Index generally had negative values for nearly all of the 
immersion components. These findings suggest that the implementation indices are relatively effective in 
discriminating higher and lower implementing schools. Still, there are exceptions to the prevailing trends. 
Some schools, such as MS 3, had generally higher implementation values for most of the indicators 
except Professional Development (-0.28). This suggests that professional development for teachers was a 
lower priority at this school in the fourth year. MS 4 had generally high levels of school support and 
Classroom Immersion, but students had a low score for Student Access and Use (-0.99) because they were 
not allowed to use their laptops at home for learning. In other schools, such as MS 17 and MS 18, 
students reported higher levels of technology access and use even though strong implementation supports 
were not in place, and their teachers’ levels of Classroom Immersion were low. 

Campus-level results for the Implementation Index displayed in Figure 3.12 illustrate the variation in the 
levels of Technology Immersion for the 21 middle schools in the fourth project year. Results for the 
Implementation Index combined with evidence from standards-based scores suggest that about a quarter 
of middle schools (6), with Implementation Index scores ranging from 0.39 to 2.58 standard deviations 
above the mean, had a stronger presence of the components of Technology Immersion compared to other 
schools, and thus a higher level of implementation that more nearly approximated expected standards. 



MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS MS 
21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 


□ Implementation Index, Year 4 


Figure 3.12. Campus means for 21 immersion middle schools (MS) on the Technology Immersion 
Implementation Index (standardized scores [z scores] with a mean of 0 and a standard deviation of 1.0). 

Conclusions 

This chapter described the components of Technology Immersion, as defined by the TEA and 
operationalized through Technology Immersion packages. Over three project years, we measured 
implementation using a two-part approach: (a) designation of standards defining four levels of immersion 
(i minimal , partial, substantial, and full), and (b) calculation of standardized implementation indices 
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(z scores). Both types of scores provide evidence relative to the strength of supports for immersion, and 
the extent of teachers’ classroom immersion and students’ technology access and use. Major findings arc 
the following. 

• Mean immersion standard scores revealed small yearly increases across most of the 
implementation support components (Leadership, Teacher Support, Parent/Community Support, 
and Professional Development) as well as increases in teachers’ overall level of Classroom 
Immersion. In contrast, the level of Student Access and Use declined across years. 

• Despite improvements, mean fourth-year immersion standard scores (ranging from 2.69 to 3. 19) 
showed that many schools needed stronger supports, especially in the areas of parent and 
community support for technology use, technical supports that addressed obstacles to technology 
use, and professional development for teachers. 

• Consistent with the second and third project years, core-subject teachers at a majority of schools 
reported only partial levels of Classroom Immersion in the fourth year. Teachers’ mean scores at 
a fifth of schools, however, revealed substantial levels of Classroom Immersion. 

• Asa whole, the standards-based implementation scores for Classroom Immersion increased 
slightly across years (from 2.48 to 2.69). Standard scores for four of the five elements of 
Classroom Immersion showed somewhat stronger implementation in the fourth year, with the 
largest increase for teachers’ use of technology for Professional Productivity and the smallest 
change for Technology Integration. The frequency with which teachers had students in their 
classrooms use technology for learning activities (Student Activities) remained relatively stable 
across years. 

• Students’ access to and use of laptops for learning within and outside of school continued to fall 
well short of expectations in the fourth year. The percentage of schools with partial levels of 
access and use decreased across years (76%, 68%, and 57%), while the percentage of schools 
with minimal access and use increased (24%, 32%, and 43%). 

• Students’ opportunities to use their laptops for learning both within classrooms and outside of 
school were affected by several factors, including mainly time lost for repairs due to aging 
laptops, schools that opted to transfer laptops from individual students to carts or classroom sets, 
schools that restricted students’ use of laptops outside of school, and teachers’ preferences 
regarding laptop use. Year-to-year comparisons indicated that the mean implementation level for 
Laptop Access Days increased between the third and fourth project years (from 2.50 to 2.64), 
whereas the yearly mean implementation levels for Core-Content Learning (2.07, 2.12, and 1.95) 
and Home Learning (1.75, 1.84, and 1.63) decreased across years. 

• Implementation indices (z scores) described each school’s level of implementation for the 
components of Technology Immersion. Fourth-year evidence from immersion standard scores 
and the Implementation Index, a composite score measuring the overall presence of immersion 
components, indicated that about a quarter of middle schools (6) had a much stronger presence of 
the immersion components compared to other schools. Thus, these schools had a higher level of 
immersion that more nearly approximated expected implementation standards. 

Despite low levels of implementation at many campuses, report chapters to follow demonstrate that 
Technology Immersion can positively affect teachers and students in many ways even at lower levels of 
implementation. 
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4. Effects of Technology Immersion on Teachers and Teaching 


In the theoretical model, researchers posited that high quality implementation of Technology 
Immersion would lead to teachers who have greater technology proficiency, use technology more for 
their own professional productivity, hold a more favorable pedagogical orientation toward technology, 
and collaborate more often with their peers to advance teaching and learning through technology. 
Moreover, teachers in schools that achieve higher levels of school and classroom immersion will have 
students who use technology more often in their classrooms and will use laptops as a tool to increase 
the intellectual challenge of lessons. 

Contrary to expectations, results reported in Chapter 3 revealed that school-level supports for 
Technology Immersion generally did not meet full implementation standards, and accordingly, 
teachers at many treatment schools reported just partial levels of Classroom Immersion. Additionally, 
as noted in the methodology chapter, control schools began to plan for Technology Immersion in the 
third year, and in the fourth year, most of the control teachers had personal laptops, digital teaching 
and learning resources, and opportunities for technology-related professional development, and 
students in many control classrooms had wireless laptops. Recognizing the less-than-ideal 
experimental conditions, we have investigated the effect of Technology Immersion on treatment 
teachers, given that the fidelity of implementation varied across schools and control teachers benefited 
from many elements of the treatment. 

Findings on the effects of immersion on teacher-mediating variables come from online surveys of 
teachers completed in fall 2004 ( N = 1,271) and again in spring 2005 ( N - 1,144), 2006 ( N = 1,175), 
2007 ( N - 1,208), and 2008 ( N = 1,159). Response rates ranged from 87% to 98% across survey 
administrations, with only small differences between comparison groups. Teachers responded to 
survey items measuring seven variables pertinent to their technology knowledge and skills 
(Technology Proficiency and Professional Productivity), ideological views (Technology Integration, 
Learner-Centered Instruction, and Resistance to Integration), frequency of student activities with 
technology (Student Classroom Activities), and interactions with peers on technology issues 
(Collaboration). Cronbach’s alpha reliability coefficients for the scale scores ranged from 0.66 to 0.98. 
(See Appendix C for technical details.) 

Researchers also conducted classroom observations during site visits at each of the treatment and 
control schools to gather information on instructional practices and changes across time. Classroom 
observations focused incrementally on the teachers of Cohorts 1, 2, and 3 students. We conducted 
observations in a sample of sixth-grade classrooms in fall 2004 and spring 2005, sixth- and seventh- 
grade classrooms in spring 2006, and sixth-, seventh-, and eighth-grade classrooms in spring 2007 and 
2008. 


Teacher Mediating Variables — HLM Analysis 

An advantage of a longitudinal study is the potential to study the nature of teacher change. The 
development of hierarchical linear models (HLM) has provided statistical tools for studying rates of 
change using measurements from multiple time points (Raudenbush & Bryk, 2002). For this study, we 
measured teacher variables on five occasions (fall 2004 through spring 2008). Our analytical sample 
included 2,137 teachers who taught at schools at some point during four implementation years, with 
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1,046 in 21 Technology Immersion schools and 1,091 in 21 control schools. Thus, we included 
teachers in the analyses even if they were not measured at all five time points. Because multilevel 
regression models do not assume equal numbers of observations (i.e., occasions of measurement), 
respondents with missing data can remain in the analysis (Hedeker, 2004; Hox, 2002). HLM, however, 
requires complete data at the teacher and school levels, so teachers were omitted if, for example, they 
were missing demographic information such as ethnicity. Our analytic approach mitigated problems 
associated with the substantial loss of teachers from analyses due to generally high teacher attrition 
rates each year of the study and varying teacher turnover rates across schools. For example, while the 
overall annual average teacher turnover rate ranged from 14% to 16%, individual school annual 
turnover rates varied from about 6% to about 42%. 

The analyses that follow contrast immersion and control teachers’ individual growth trajectories for 
each of the seven scales described above. We analyzed effects using three-level hierarchical growth 
models. FILM growth models produce teacher- and school-specific effects (i.e., the extent which the 
survey scores vary across time, teachers, and schools). In our models, we hypothesize that school 
poverty is related to teachers’ initial status and yearly growth rate. This supposition stems from an 
investigation of the implementation of Technology Immersion indicating that a higher concentration of 
economically disadvantaged students in a school is negatively associated with stronger levels of 
school and classroom immersion. Similarly, other research reviews confirm the negative effects of 
school poverty on school reform efforts (Desimone, 2002) and student achievement (Sirin, 2005). 

Since Technology Immersion Pilot (TIP) grants targeted high-needs schools, the percentages of 
disadvantaged students were generally high across most of the study’s schools. Even so, school 
poverty concentrations varied substantially (ranging from 31% to 100%). The statistical model is 
described below. 

Level 1 : Repeated-Measures Model 

Level 1 is a repeated-measures model (i.e., survey time within teachers) that enables us to capture key 
features of growth (e.g., initial status, rate of change). In the model, Y tlJ is the survey scale score at year 
t for teacher i in school j. Survey Time is the point at which teachers completed the online surveys 
(0=fall 2004, I =spring 2005, 2=spring 2006, 3=spring 2007, 4=spring 2008). The key parameters in 
the model are 7r w , and n U j. The coefficient jtoy represents the “initial status” (that is, the initial survey 
scale score) for teacher / in school j in fall 2004, and n tj] is the growth rate (rate of change) for teacher i 
in school j per school year. The e,y is the error term (within-teacher measurement error) assumed to be 
normally distributed with a mean of 0 and a constant variance. Thus, at level 1 the model is 

Y ti j = K 0i j + Klij (Survey Time) tij + e tij . 

Level 2: Teacher-Level Model 

The Level 2 model (between-teachers model) allows us to determine differences between teachers in 
features of growth (e.g., initial status, rate of change). In the teacher-level model, noy is the teacher’s 
initial survey scale score and n ti j is the teacher’s rate of growth per school year. In the model, p 0 oj 
represents the mean initial status within school j, and p 10j is the mean yearly rate of teacher change 
within school j. The r 0 ij and r H] are residuals (i.e., random effects). At level 2, the model is 

Hoy - fiooj + r 0i j 

rtlij = filOj + t'lij. 
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Level 3: School-Level Model 


At the school level (level 3), we examined how teachers’ initial status and growth varied across 
schools as a function of school- level random effects (ju 0 oj and // 10 j) as well as school conditions, 
including immersion status and school poverty. That is, we hypothesized that being in an immersion 
school is positively related to teachers’ growth on technology-related scores, after controlling for the 
poverty level of the school. Thus, we pose the following school-level model: 

Pooj = y 0 oo + Yooi( Immersion Status )j + yoo 2 (School Poverty )j + fiooj 

fiioj — yioo + yioi( Immersion Status)j + y I0 2 (School Poverty)j + /Liioj. 

In the model, fiooj is the mean initial status for teachers in school j and y 0 oo is the overall mean 
initial status (grand mean); f} 10 j is the mean teacher growth rate in school j and y 100 is the overall 
mean teacher growth rate. Immersion status is an indicator variable with a value of 0 for a control 
school and a value of 1 for an immersion school. School poverty is a continuous variable with 
percentages ranging from ranging from 31% to 100%, with a mean of 68.5%. The coefficients 
yooi and y m represent the direction and strength of association of immersion status and school- 
level initial status. 


Effects of Immersion on Teachers 

After adjusting for school poverty. Technology Immersion had a statistically significant effect on 
teachers’ rates of growth for four of seven technology-related variables (Table 4.1). This was a notable 
change from the significant effect on teachers’ growth for six technology-related variables in the first 
through third project years. Teachers at Technology Immersion schools in the fourth year, on average, 
had significantly steeper growth trends than teachers at control schools for Technology Proficiency 
and Professional Productivity, one measure of teachers’ ideology (Learner-Centered Instruction), and 
the frequency of Student Classroom Activities (with technology). In contrast to previous results, there 
was no significant difference between the treatment and control teachers for measures of their growth 
in Technology Integration or Collaboration with their colleagues. 

Table 4.1. Immersion Effects on Estimated Mean Growth Rates for Teacher Variables 

Statistics for Teachers in Immersion Schools 
with Average School Poverty c 



Immersion 
Effect Net of 
School 
Poverty 

Average 
Estimated 
Initial Status 
Fall 2004 

Yearly 
Growth Rate 

Average 

Estimated 

Score 

Spring 2008 

Yearly 
Growth Rate 
for Control 
Teachers 

Technology Proficiency 3 

Yes 

4.53 

q 29*** 

5.69 

0.15*** 

Professional Productivity 13 

Yes 

2.95 

0.18*** 

3.67 

Q J]*** 

Ideology 






Technology Integration 3 c 

No 

3.29 

0.40 

4.89 

0.30 

Learner-Centered Instruction 3 

Yes 

3.72 

0.31*** 

4.96 

0.20*** 

Resistance to Integration 3 

No 

2.17 

0.04 

2.33 

0.01 

Student Classroom Activities' 3 

Yes 

2.02 

Q J7*** 

2.70 

q Q7*** 

Collaboration 13 

No 

2.44 

0.07 

2.72 

0.06*** 


Source: Online teacher surveys conducted in fall 2004 and spring 2005, 2006. 2007, and 2008. 
*p < .05. **p < .01. ***p < .001. 

‘‘Items measured on a 7-point scale. b Items measured on a 5-point scale. 

c Controls for the effect of differences in initial status on the Technology Integration growth rate. 
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Control teachers, who were exposed to elements of the Technology Immersion model in the fourth 
year, had significantly positive growth trends for five technology-related variables. Although control 
teachers’ yearly growth rates were significantly less steep than rates for immersion teachers, the 
introduction of technology resources in control schools had a positive effect on teachers’ Technology 
Proficiency, use of technology for Professional Productivity, affiliation with Learner-Centered 
Instructional ideologies, and the frequency of technology-related Student Classroom Activities and 
Collaboration with peers. Sections to follow explain the nature of teacher change, with Tables 4.2, 4.3, 
and 4.4 providing school-level statistics for the HLM analyses of immersion effects. 

Technology Knowledge and Skills 

Texas Technology Applications Standards require all teachers to master and use technology-related 
terminology, concepts, and strategies, and to use tools to accomplish a range of tasks (e.g., 
communicate with diverse audiences and analyze electronic information). Thus, our online surveys 
included measures of teachers’ Technology Proficiency and Professional Productivity. For 
Technology Proficiency, teachers rated their skills in using various technology applications on a 7- 
point scale ranging from 1 ( not true of me now) to 7 (very true of me now). The proficiency scale 
included items measuring technology operations (e.g., send email to coworkers, parents, or peers; 
search for and find a Web site; find primary sources of information on the Internet) and items related 
to classroom instruction (e.g., using the computer for presentations or creating a lesson plan or unit 
incorporating technology). 

HLM statistics in Table 4.2 show that immersion teachers grew in technology proficiency at a 
significantly faster rate (0.29 scale-score point per year) than control teachers (0.15 point per year). 
Immersion teachers began with slightly lower mean proficiency scores than control teachers in fall 
2004, but they surpassed control teachers in spring 2005 and continued to widen the proficiency gap 
during the next three school years. 


Table 4.2. Immersion (Fixed) Effect Analyses for Teacher Technology 
Knowledge and Skills Variables 


Dependent variable 
and predictor 

Technology 

Proficiency 

Professional 

Productivity 

Gamma 

Coefficient 

t-value 

Gamma 

Coefficient 

t-value 

Initial status (fall 2004) 

4.692 

58.26*** 

3.011 

51.53*** 

Immersion 

-0.165 

-1.48 

-0.062 

-0.80 

School Poverty 

0.001 

0.16 

0.001 

0.57 

Growth rate 

0.147 

10.53*** 

0.110 

Y2 17 *** 

Immersion 

0.138 

6.06*** 

0.069 

4 92*** 

School Poverty 

-0.002 

-3.23** 

0.000 

-0.66 


f < .10. *p < .05. **p < .01. **-*p < .001. 


Similar to previous years, teachers who taught at immersion and control schools with higher levels of 
school poverty (percentages of economically disadvantaged students) had significantly slower rates of 
growth for Technology Proficiency. For each percentage point increase in school poverty, teachers had 
a 0.002 scale-score decrease in proficiency. Thus, a 20% decrease in school poverty predicted a 0.04 
point increase in teachers’ yearly growth in proficiency (i.e., 20 x 0.002); a 20% increase in school 
poverty predicted a 0.04 point decrease in teachers’ yearly growth. As the level of school poverty 
increases, the teacher proficiency gap widened. 
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Teachers also rated the frequency with which they used technology for Professional Productivity on a 
5-point scale ranging from 1 (never) to 5 ( almost daily). Productivity items, for example, measured 
teachers’ use of technology for administrative, classroom management, communication, and 
instructional puiposes. Similar to findings for Technology Proficiency, teachers at immersion schools 
had significantly steeper rates of growth than control teachers in the use of technology to improve their 
productivity. The estimated yearly mean growth trajectories for immersion and control teachers in 
schools with average poverty were 0. 1 8 and 0.11 scale-score points per year, respectively. Teachers 
working in schools with higher percentages of disadvantaged students grew in productivity at similar 
rates. 

Figure 4. 1 compares the growth in technology knowledge and skills for treatment and control teachers. 
As the figure illustrates, immersion teachers grew in Technology Proficiency and Professional 
Productivity at a faster rate than control teachers, but control teachers had a significantly positive 
growth trend for both of the technology competency indicators. 



— ■ — Immersion: Productivity (5-pt) — — Control: Productivity (5-pt) 

Figure 4.1. Estimated mean growth trajectories for treatment and control teachers working in 
schools with average levels of school poverty on Technology Proficiency and Professional 
Productivity indicators (ratings on either 5-point or 7-point scales). 

Ideology 

Teachers also responded to items measuring their ideological views relative to technology integration 
and constructivist practices on a 7-point scale, ranging from 1 (not true of me now) to 7 (very true of 
me now). Items from the Levels of Technology Implementation (LoTi) Questionnaire (Moersch, 2001) 
measured three latent variables (Technology Integration, Learner-Centered Instruction, and Resistance 
to Integration). FILM results detailed in Table 4.3 and illustrated in Figure 4.2 show that at both 
immersion and control schools, teachers on average became more positive towards innovative 
technology practices across time. 

The Technology Integration scale included items gauging teachers’ actions supporting curricular and 
instructional infusion of technology. For example, teachers indicated the extent to which computer- 
related activities enabled them to support students’ authentic problem solving or to promote critical 
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thinking. Coefficients reported in Table 4.3 show that teachers in immersion schools had a positive 
rate of change for Technology Integration but the growth rate was not significantly steeper than the 
growth of control teachers. The mean estimated growth trajectory for immersion teachers who worked 
in schools having average levels of school poverty was 0.40 scale-score point per year compared to 
0.30 scale-score point for control teachers. This result differed from results for previous study years 
showing that teachers at Technology Immersion schools grew significantly faster than control teachers 
in practices supporting Technology Integration. 


Table 4.3. Immersion (Fixed) Effect Analyses for Teacher Ideology Variables 


Dependent variable 
and predictor 

Technology 

Integration 11 

Learner-Centered 

Instruction 

Resistance 
to Integration 13 

Gamma 

Coefficient 

t- value 

Gamma 

Coefficient 

t- value 

Gamma 

Coefficient 

£-value 

Initial status (fall 2004) 

2.847 

38.95*** 

3.683 

56.37*** 

2.463 

44.57*** 

Immersion 

0.445 

4.61*** 

0.036 

0.39 

-0.295 

_4 24*** 

School Poverty 

0.010 

3.50** 

0.006 

2.62* 

-0.003 

-1.75f 

Growth rate 

0.002 

0.00 

0.199 

11.75*** 

0.011 

0.86 

Immersion 

0.105 

1.14 

0.110 

3.42** 

0.024 

1.03 

School Poverty 

-0.005 

-2.30* 

-0.002 

-2.08* 

0.002 

3.69** 

Initial status 

0.103 

0.55 

- 

- 

- 

- 


f p < .10. *p < .05. **p < .01. ***p < .001. 

“"Technology Immersion teachers had significantly higher initial Technology Integration scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, indicated that the immersion effect was 
non-significant after controlling for initial differences. Thus, the coefficients from the latent variable regression model are 
reported here. 

b Technology Immersion teachers had significantly lower initial Resistance to Integration scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, indicated that the immersion effect was a 
non-significant predictor of the growth rate with and without controlling for initial differences. Thus, the coefficients from 
the original growth model are reported here. 

Consistent with previous years, teachers at immersion schools compared to control grew at a 
significantly faster rate in their affiliations with principles of Learner-Centered Instruction. Across 
survey administrations, immersion teachers reported increasingly higher ratings for items describing 
pedagogical practices such as having students establish individual learning goals, emphasizing 
experiential learning, and providing real-world experiences. The estimated yearly growth in the 
adoption of learner-centered practices for immersion and control teachers in schools with average 
poverty was 0.31 and 0.20 scale-score points, respectively. Teachers in schools with higher 
concentrations of school poverty had significantly slower rates of growth relative to both technology 
integration and learner-centered practices. 

For the Resistance to Integration scale, teachers expressed their strength of association with items 
suggesting that classroom computers are not a priority, are not a necessary paid of instruction, and arc 
not practical for students. Contrary to the two ideological indicators discussed above, there was little 
change in the growth rate on the Resistance to Integration scale for either immersion or control 
teachers. Scores indicated that teachers, on average, expressed a relatively low level of resistance to 
technology integration, and their level of resistance remained fairly constant across years. Still, 
teachers in schools with higher levels of student poverty expressed significantly greater resistance to 
technology. 


38 




6i 



1 T 1 1 1 1 1 

Fall 2004 Spring 2005 Spring 2006 Spring 2007 Spring 2008 

Survey Time 

— A — Immersion: Technology Integration Control: Technology Integration 

— B — Immersion: Learner-Centered Control: Learner-Centered 

— • — Immersion: Resistance Control: Resistance 

Figure 4.2. Estimated mean growth trajectories for treatment and control teachers working in 
schools with average levels of school poverty on Ideology indicators: Technology Integration, 
Learner-Centered Instruction, and Resistance to Integration (ratings on 7-point scales). 

Student Classroom Activities and Teacher Collaboration 

Table 4.4 provides HLM statistics for measures of teachers’ classroom activities and collegial 
collaboration. The Student Classroom Activities scale provided an estimate of the frequency — on a 5- 
point scale ranging from 1 (never) to 5 ( almost daily ) — with which teachers had students in their 
typical class use technology in various ways. For example, teachers might have students use 
technology for writing, learning and practicing skills, communication, or Internet research. As 
expected, given the greater availability of laptops at immersion schools, teachers at treatment schools 
had a significantly faster growth rate for Student Classroom Activities (0.17 and 0.07 scale-score 
points per year, respectively, for immersion and control teachers in schools with average poverty). 
School poverty was a significantly negative predictor of teachers’ growth in the frequency of students’ 
classroom activities involving technology. 

Even though treatment teachers had their students use technology in classrooms more frequently 
across years, estimated mean scores displayed in Figure 4.3 show that by spring 2008, teachers, on 
average, had students use various technology applications in their classes infrequently (about once or 
twice a month, M = 2.72). Students in schools with higher concentrations of economically 
disadvantaged students used technology even less often. Students in control teachers’ classrooms, on 
average, used technology applications a few times a year (M - 2.15 in spring 2008). 
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Table 4.4. Immersion (Fixed) Effect Analyses for Student Classroom 
Activities and Teacher Collaboration Variables 


Dependent variable 
and predictor 

Student Classroom 
Activities 2 

Teacher 

Collaboration 11 

Gamma 

Coefficient 

Lvalue 

Gamma 

Coefficient 

Lvalue 

Initial status (fall 2004) 

1.858 

39.52*** 

2.292 

48.04*** 

Immersion 

0.161 

2.65* 

0.143 

2.29* 

School Poverty 

0.004 

2.56* 

0.004 

2.30* 

Growth rate 

0.073 

5.90*** 

0.055 

3 97 *** 

Immersion 

0.101 

5.60*** 

0.018 

0.95 

School Poverty 

- 0.001 

-2.51* 

0.000 

0.15 


t p < .10; *p < .05; **p < .01; ***p < .001. 

“Treatment teachers had significantly higher initial scores for student activities and 
collaboration. Latent variable regressions, controlling for the effects of initial differences 
on growth rates, indicated that immersion was a significant predictor of growth rates with 
and without controlling for initial differences. Thus, the coefficients from the original 
growth models are reported here. 
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— • — Immersion: Student Activities Control: Student Activities 

Figure 4.3. Estimated mean growth trajectories of treatment and control teachers working in 
schools with average levels of school poverty for Student Classroom Activities (ratings on 5- 
point scale). 

We also reasoned that a greater abundance of technology resources and opportunities for shared 
professional development would lead to stronger teacher connections. Accordingly, the Collaboration 
scale measured teacher interactions with colleagues that supported improvements in instructional 
practices, such as coaching and mentoring, collectively developing technology lessons, and 
exchanging information about their students. Contrary to previous years, there was no significant 
immersion effect on teachers’ yearly growth rate for Collaboration. Immersion and control teachers 
had si mi lar growth trends relative to collegial interactions (0.07 and 0.06 scale-score points, 
respectively). Campus poverty had a negligible association with teacher collaboration. 
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Effects of Immersion on Classroom Practice 


To further understand teachers’ instructional practices, researchers conducted classroom observations 
in samples of core-subject classrooms (reading/English language aits, mathematics, science, and social 
studies). We added teachers at higher grade levels each year so that the observed teachers taught 
students included in the three cohorts of students followed in the study. In fall 2004 and spring 2005, 
we observed sixth-grade teachers. The classroom sample included observations of sixth- and seventh- 
grade teachers in spring 2006, and observations of sixth-, seventh-, and eighth-grade teachers in spring 
2007 and 2008. Each year, we purposefully selected teachers to represent grade levels and subject 
areas, and when possible, we selected teachers who had been observed in previous years. 

Classroom observations involved either single observers (about 75% of classrooms) or pairs of 
observers (about 25% of classrooms). Paired observations permitted the calculation of inter-observer 
agreement. In fall 2004, researchers observed 125 classrooms (60 treatment and 65 control) in half of 
the schools. Subsequently, we conducted observations in all schools. We observed 206 classrooms in 
2005 (105 treatment and 101 control), 217 classrooms in 2006 (114 treatment and 103 control), 194 
classrooms in 2007 (95 treatment and 99 control), and 230 classrooms in 2008 (117 treatment and 113 
control). At small campuses, researchers observed nearly all core-content teachers; at larger campuses, 
we observed a representative sample of classrooms. 

Across data-collection periods, observations at treatment and control schools included nearly equal 
proportions of teachers by subject-area taught, gender, highest degree earned, and years teaching 
experience. Observations included somewhat more English language arts and reading teachers (28% to 
33% of observed teachers), and somewhat fewer mathematics teachers (22% to 29%), social studies 
teachers (19% to 27%), and science teachers (13% to 24%). Variations reflected our interest in 
documenting the instructional practices of teachers whose students were included in cohorts being 
tracked across years and their TAKS-tested subject areas. 

During observations, data collectors used the Observation of Teaching and Learning (OTL) instrument 
to record descriptive information about the classroom environment, and to make time-interval ratings 
for classroom organization, teacher activities and technology use, student activities and technology 
use, student engagement, and student collaboration. Observers also recorded notes during the 
observations to capture the lesson’s content focus and objectives, teachers’ questioning strategies 
(lower and higher order), and students’ learning experiences. Following classroom observations, 
observers used time-interval ratings and descriptive notes to rate the Intellectual Challenge of 
classroom work (rating scales developed by Newmann, Secada, & Wehlage, 1995). One section of the 
OTL included 5-point rating scales for four standards of the intellectual quality of instruction: 

• Construction of Knowledge: Higher Order Thinking. Instruction involves students in 
manipulating information about ideas by synthesizing, generalizing, explaining, 
hypothesizing, or arriving at conclusions that produce new meaning and understanding. 

• Disciplined Inquiry: Deep Knowledge. Instruction addresses central ideas of a topic or 
discipline with enough thoroughness to explore connections and relationships and to produce 
relatively complex understandings. 

• Disciplined Inquiry: Substantive Conversation. Students engage in extended conversational 
exchanges with the teacher or peers about subject matter in a way that builds an improved and 
shared understanding of ideas or topics. 

• Value Beyond School: Connections to the World Beyond the Classroom. Students make 
connections between knowledge and either public problems or personal experience (Newmann 
et al„ 1995). 
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An aggregate score across three of the four standards was used as an overall measure of the 
Intellectual Challenge of instruction for each teacher. The score for Substantive Conversation was 
omitted from the composite score because ratings were highly influenced by the organizational 
structure of lessons. Specifically, lessons involving teacher-directed discussions typically yielded more 
public conversations, and thus, better opportunities to gather evidence on conversational exchanges 
than lessons with students working in small groups or individually. Additionally, to enhance observer 
agreement for OTL ratings, we conducted training sessions for researchers immediately before each 
series of site visits began, except for the fourth year. Across years, we utilized Many-Facet Rasch 
Measurement (Linacre, 2004) to adjust the measure of Intellectual Challenge for the relative severity 
(or leniency) of each observer during analyses. 

Table 4.5 reports the adjusted composite Intellectual Challenge scores for immersion and control 
teachers across five data-collection periods. When researchers conducted baseline observations in fall 

2004, sixth-grade control teachers’ mean Intellectual Challenge score (1.88) was significantly higher 
than immersion teachers’ instructional score (1.62). The difference represented a moderate effect size 
(ES = -0.33) favoring control teachers. Thus, control teachers initially engaged students in lessons that 
required a higher level of thinking, delved into topics more thoroughly, and made stronger connections 
with students’ background experiences and the world beyond the classroom. On the contrary, in spring 

2005, sixth-grade teachers’ lessons at immersion schools received a slightly higher mean Intellectual 
Challenge score (1.87) than control teachers’ instruction (1.81). The difference between the groups, 
however, was statistically insignificant. 


Table 4.5. Adjusted Intellectual Challenge Scores for Treatment and Control Teachers 



Treatment 

Control 



Effect 

Group 

N 

Mean 

SD 

N 

Mean 

SD 

1- value 

P 

Size 

Fall 2004 

60 

1.62 

0.71 

65 

1.88 

0.87 

-1.84 

t"- 

o 

d 

-0.33 

Spring 2005 

106 

1.87 

0.93 

101 

1.81 

0.90 

0.48 

0.63 

0.07 

Spring 2006 

114 

1.82 

0.75 

103 

1.77 

0.76 

0.47 

0.64 

0.07 

Spring 2007 

95 

2.06 

0.80 

99 

1.91 

0.77 

1.28 

0.20 

0.19 

Spring 2008 

117 

2.06 

0.78 

113 

2.00 

0.76 

0.59 

0.56 

0.08 


Notes. Observations at 21 immersion and 21 control schools. Intellectual Challenge of Instruction scores could 
range from 1 (low challenge) to 5 (high challenge). The rating for Substantive Conversation was deleted from the 
composite score. Difference is statistically significant at the 0.10 level. Effect size is Cohen's d. 


In spring 2006, lessons observed in sixth- and seventh-grade teachers’ classrooms at immersion 
schools received a slightly higher mean Intellectual Challenge score (1.82) than control teachers’ 
lessons (1.77), but not by a statistically significant margin. In spring 2007, lessons delivered by sixth-, 
seventh-, and eighth-grade teachers at immersion schools had a notably higher mean level of 
Intellectual Challenge (2.06) compared to control teachers’ instruction (1.91). Although the difference 
between groups was statistically insignificant, the small, positive effect size (0.19) showed that 
instruction at immersion schools was a bit more challenging. In spring 2008, differences between the 
immersion and control teachers narrowed. The lessons observed in sixth through eighth grade 
classrooms at immersion schools received a slightly higher mean Intellectual Challenge score (2.06) 
compared to control teachers’ lessons (2.00). The difference between groups was statistically 
insignificant and the effect size was very small (0.08). 

Table 4.6 su mm arizes findings across the data collection periods for each of the Intellectual Challenge 
domains. Effect sizes show that control teachers’ instruction in fall 2004, compared to immersion 
teachers, had a higher mean level of intellectual challenge for each of the four standards. Flowever, in 
spring 2005, immersion teachers’ lessons received higher ratings for Fligher Order Thinking (ES = 
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0.18) and Depth of Knowledge (ES = 0.09). In spring 2006 and 2007, immersion teachers’ lessons, 
compared to control teachers, received higher Intellectual Challenge scores for each of the four 
standards. Effect sizes indicated that immersion teachers had a greater instructional emphasis on 
Higher Order Thinking (0.22 in 2006, 0.28 in 2007) and Connections beyond the Classroom (0.06 in 
2006, 0.18 in 2007). In 2008, differences between the intellectual rigor of immersion and control 
teachers’ lessons resembled findings for spring 2005 (the end of the first implementation year). 
Although group differences were statistically insignificant, immersion teachers’ instruction had a 
greater emphasis on Higher Order Thinking (ES = 0.22) and Depth of Knowledge (ES = 0.19), but 
control teachers’ lessons involved more Substantive Conversation (ES = -0.04) and Connections 
Beyond the Classroom (ES = -0.19). 


Table 4.6. Adjusted Intellectual Challenge Scores for Immersion and Control Teachers, by 
Dimension and Year 


Standard 

Treatment 

Control 



Effect 

Mean 

SD 

Mean 

SD 

t-value 

P 

Size 

Fall 2004 (Baseline) 

Higher Order Thinking 

1.67 

1.02 

1.80 

1.03 

-0.73 

0.470 

-0.13 

Depth of Knowledge 

1.60 

0.94 

1.85 

1.05 

-1.38 

0.171 

-0.25 

Substantive Conversation 

1.33 

0.77 

1.40 

0.75 

-0.49 

0.625 

-0.09 

Connections Beyond the Classroom 

1.35 

0.66 

1.48 

0.83 

-0.94 

0.349 

-0.17 

Spring 2005 

Higher Order Thinking 

1.89 

1.04 

1.71 

1.00 

1.21 

0.227 

0.18 

Depth of Knowledge 

1.83 

1.07 

1.73 

1.06 

0.65 

0.518 

0.09 

Substantive Conversation 

1.40 

0.74 

1.44 

0.84 

-0.36 

0.720 

-0.05 

Connections Beyond the Classroom 

1.79 

1.01 

1.82 

1.05 

-0.22 

0.827 

-0.03 

Spring 2006 

Higher Order Thinking 

1.91 

0.93 

1.71 

0.90 

1.64 

0.104 

0.22 

Depth of Knowledge 

1.85 

0.88 

1.83 

0.97 

0.13 

0.899 

0.02 

Substantive Conversation 

1.46 

0.73 

1.45 

0.92 

0.09 

0.932 

0.01 

Connections Beyond the Classroom 

1.63 

0.91 

1.58 

0.85 

0.41 

0.681 

0.06 

Spring 2007 

Higher Order Thinking 

2.20 

1.05 

1.92 

0.92 

1.98 

0.049* 

0.28 

Depth of Knowledge 

2.15 

0.98 

2.01 

0.98 

0.97 

0.332 

0.14 

Substantive Conversation 

1.46 

0.77 

1.38 

0.70 

0.75 

0.452 

0.11 

Connections Beyond the Classroom 

1.75 

0.97 

1.58 

0.88 

1.29 

0.198 

0.18 

Spring 2008 

Higher Order Thinking 

2.27 

1.06 

2.04 

1.00 

1.69 

0.092 

0.22 

Depth of Knowledge 

2.32 

1.02 

2.14 

0.93 

1.43 

0.156 

0.19 

Substantive Conversation 

1.40 

0.74 

1.50 

0.72 

-0.97 

0.332 

-0.04 

Connections Beyond the Classroom 

1.65 

0.90 

1.83 

1.00 

-1.45 

0.148 

-0.19 


Note. Rating scales developed by Newmann, Secada, & Wehlage (1995) ranged from 1 to 5. Teacher counts: fall 2004 (60 
immersion and 65 control), spring 2005 (105 immersion and 101 control), spring 2006 (114 immersion and 103 control), 
spring 2007 (95 immersion and 99 control), and spring 2008 (117 immersion and 113 control). 

*Statistically significant difference. Effect size is Cohen's d. 

In general, the introduction of technology resources (the treatment) into control teachers’ classrooms 
has biased measures of differences between treatment and control groups. Nevertheless, longitudinal 
effect size trends suggest that the intellectual rigor of treatment teachers’ instruction has improved 
somewhat over time. The introduction of technology resources in control classrooms seems to have 
had a similarly positive influence on control teachers’ instruction. Multi-year findings from 
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observations in middle-school classrooms also raise concerns about the intellectual rigor of students’ 
assigned tasks. Results for all observed classrooms indicated that lessons in middle-school core classes 
generally failed to intellectually challenge students, with average ratings about 2.3 or less on the 5- 
point intellectual challenge of instruction scales. 

Conclusions 

We found that working in treatment schools that had been implementing Technology Immersion 
across four school years had a significantly positive effect on teachers’ growth in a number of areas. 
Similarly, the acquisition of immersion resources such as teacher laptops, digital resources, 
professional development, and student laptops during the past two school years spurred control 
teachers’ growth in technology competency and use. Key findings are the following: 

• Immersion teachers grew in Technology Proficiency and in their use of technology for 
Professional Productivity at significantly faster rates than control teachers. 

• Immersion teachers expressed increasingly stronger ideological affiliations with Learner- 
Centered Instruction than control teachers. At the same time, immersion teachers reported 
generally low Resistance to Integration. 

• Students in immersion classrooms used technology applications significantly more often than 
control students for core-subject learning activities (Student Classroom Activities). 

• Across both treatment and control campuses, school poverty was negatively associated with 
teachers’ growth on several technology-related indicators. Notably, teachers in schools with 
above average levels of school poverty grew in Technology Proficiency at a significantly 
slower rate, and expressed significantly weaker affiliations with Technology Integration and 
Learner-Centered Instruction, and stronger Resistance to Integration. Teachers in schools with 
greater school poverty also had their students use technology applications significantly less 
often in their classrooms (Student Classroom Activities). 

• The introduction of elements of the Technology Immersion model in control schools had a 
significantly positive effect on control teachers. In the fourth year, control teachers had 
statistically significant growth trends for Technology Proficiency, Professional Productivity, 
Learner-Centered Instruction, Student Classroom Activities (with technology), and 
Collaboration (with peers). Teachers’ growth has narrowed the gap between treatment and 
control groups. 

• Longitudinal effect size trends suggest that the availability of laptop computers and digital 
resources has allowed students in Technology Immersion schools to experience more 
intellectually demanding work. Nevertheless, ratings of the Intellectual Challenge of 
classroom instruction indicated that the intellectual demand of core-subject lessons was 
typically low across all middle-school classrooms observed. 
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5. Effects of Technology Immersion on Students and Learning 


In the theoretical model of Technology Immersion, we assumed that improved school and classroom 
environments for technology would lead to more technology-adept teachers who use technology more 
effectively for their own puiposes and have students use technology more often and for more 
intellectually challenging lessons. We also reasoned that students who experienced improved school 
and classroom conditions would acquire greater technology proficiency, use technology more often for 
learning, collaborate more often with peers, have opportunities for more rigorous and relevant school 
work, feel more strongly engaged in school and learning, and become more self-directed. Consistent 
with our suppositions, findings reported in Chapter 4 confirmed that teachers at immersion schools, in 
comparison to their control counterparts, are more technically proficient and productive, have their 
students use technology more often in class, and provide more intellectually demanding assignments. 
We investigate in this chapter the effects of Technology Immersion on students and their learning 
experiences. 


Immersion Effects on Student Mediating Variables 

Data on student mediating variables come from paper- and-pencil surveys {Student Questionnaire and 
Style of Learning Inventory ) completed by students as baseline measures in fall of their sixth-grade 
year and again as post-measures in spring of each project year. Cohort 2 (eighth graders) and Cohort 3 
(seventh graders) completed surveys in the fourth year. Cohort 1 students who attended various high 
schools as ninth graders in the fourth year did not complete surveys. The Student Questionnaire 
measured students’ technology proficiency, technology use, and views on technical problems. The 
questionnaire also gauged students’ opportunities to work with peers in small groups and their 
satisfaction with school. The Style of Learning Inventory ( SLI) measured various aspects of students’ 
self-directed learning. Overall, response rates for the Student Questionnaire were in the 80% to 90% 
range across time periods, with only slight differences in response rates between cohorts and 
comparison groups. Response rates for the SLI ranged from 77% to 89% across administrations. There 
were only slight differences in SLI response rates between cohorts and comparison groups. (See 
additional detail in the methodology chapter.) 

Immersion effects were estimated for six scales: Classroom Activities, Small-Group Work, Technical 
Problems, Technology Proficiency, Self-Directed Learning (Cohort 2 only), and School Satisfaction. 
Cronbach’s alpha coefficients (measures of internal consistency reliability) for student-level scales 
ranged from 0.77 to 0.94. (See Appendix C for details.) 

HLM Growth Analyses 

Researchers used hierarchical linear modeling (HLM) growth models to examine the effects of 
Technology Immersion on students’ individual growth rates for the six measures. For Cohort 2, we 
collected data at four time points: fall 2005 (baseline) and spring 2006, 2007, and 2008 (after students’ 
first, second, and third immersion years, respectively). For Cohort 3, we collected data at three time 
points: fall 2006 (baseline) and spring 2007 and 2008 (after students’ first and second immersion 
years). Analyses contrasted the growth trajectories for students at Technology Immersion and control 
schools. We analyzed immersion effects on students’ self -perceptions and technology-related activities 
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using three-level hierarchical linear growth models. These HLM models produced student- and 
school-specific effects (i.e., the extent to which scale scores varied across time, students, and schools). 


Level 1: Repeated-Measures Model 

Level 1 is a repeated-measures model (i.e., survey time within students) that enabled us to capture key 
features of growth (e.g., initial status, rate of change). In the model, Y nj is the survey scale score at year 
t for student i in school j, and Survey Time is the point at which students completed surveys (Cohort 2, 

0 = fall 2005, 1 = spring 2006, 2 = spring 2007, and 3 = spring 2008; Cohort 3. 0 = fall 2006, 

1 = spring 2007, and 2 = spring 2008). The key parameters in the model are jvoy and my. The 
coefficient noy represents the “initial status” (that is, the estimated initial scale score), for student i in 
school j in fall, and n lls is the annual growth rate (rate of change) for student i in school j. The e t y is the 
error term (within-student measurement error) assumed to be normally distributed with a mean of 0 
and a constant variance. Thus, at Level 1, the model is 

Y ti j - Koij + n U j (Survey Time) tiJ + e,y. 

Level 2: Student-Level Model 

The Level 2 model (between-students model) allowed us to determine differences between students in 
features of growth (e.g., initial status [;%■], rate of change |^/ y ]). In the student-level model, / 3 0 oj 
represents the mean initial status of a more advantaged student (advantaged = 0, disadvantaged =1) 
within school j, and fi W j represents the mean rate of change for an advantaged student within school j. 
The coefficients /3 0 ij and [i / /; represent the effects of student poverty on initial status and school year 
rate of change, respectively. The r 0 y and r 0i/ are residuals (i.e., random effects). At level 2, the model is 


noy = fiooj + Poij( Disadvantaged)ij + r 0ij 
ftiij = P ioj + Pnj(Disadvantaged)ij + r n j. 

Level 3: School-Level Model 

At the school level (Level 3), we examined how students’ initial status (fiooj) and growth (fiwj) varied 
across schools as a function of school- level random effects (ju 0 oj and // 10j ), as well as school conditions, 
including immersion status (an indicator variable with a value of 0 for a control school and a value of 
1 for an immersion school) and school poverty (a continuous variable with percentages ranging from 
31% to 100%, and with a grand mean of 68.5%). That is, we theorized that being in an immersion 
school was positively related to students’ growth on technology-related scores, after controlling for the 
poverty level of the school. Thus, we posed the following school-level model: 

Pooj- yooo + Yooii Immersion status )j + y 0 02( School Poverty )j + n 0 oj 
Pioj = yioo + yioi( Immersion status )j + y ^(School Poverty )j + fi 10 j. 

In the model, yooo is the overall mean initial status of an advantaged student at a control campus with 
an average level of school poverty, and y m is the overall mean student growth rate (of an advantaged 
student at a control campus with an average level of school poverty). The coefficients y 001 and y 101 
represent the direction and strength of association of immersion status on school-level initial status and 
growth rate, respectively. In addition, y 0 02 and y /02 represent the effect of school poverty on school- 
level initial status and growth rate, respectively. Analyses for Cohort 2 involved a total of 4,528 
students who were continuously enrolled in schools since October 2005, with 2,167 at immersion 
schools and 2,361 at control schools. Analyses for Cohort 3 involved 4,445 students continuously 
enrolled since October 2006, with 2,073 at immersion schools and 2,372 at control schools. 
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Immersion Effects on Technology Experiences and Self-Perceptions 


Analyses involved the estimation of three-level HLM growth models for Cohort 2 (six models) and 
Cohort 3 (five models). As Table 5.1 shows, we used separate models to estimate the effects of 
Technology Immersion on growth rates for measures of students’ school technology experiences, 
including Classroom Activities, Small-Group Work, and Technical Problems, as well as students’ self- 
perceptions of their Technology Proficiency, Self-Directed Learning (Cohort 2 only), and School 
Satisfaction. 


Table 5.1. Cohorts 2 and 3: Immersion Effects on Estimated Mean Growth Rates for Student 
Mediating Variables 


Scale Scores 

Immersion 
Effect Net 
of Student 
and School 
Poverty 2 

Immersion 
Yearly Growth Rate 

Control 

Yearly Growth Rate 

Advantaged 

Students 

Dis- 

advantaged 

Students 

Advantaged 

Students 

Dis- 

advantaged 

Students 

Cohort 2 (8th Graders) 

School Technology 

Classroom Activities (5-pt) 

Yes* 

0.19 

0.22 

0.10 

0.13 

Small-Group Work (5-pt) 

Yes** 

0.10 

0.10 

0.01 

0.01 

Technical Problems (5-pt) 

Y ^ ^ 

0.34 

0.36 

0.15 

0.16 

Student Self-Perceptions 

Technology Proficiency (5-pt) 

Yes** 

0.35 

0.35 

0.26 

0.26 

Self-Directed Learning (7-pt) 

No 

-0.13 

-0.13 

-0.14 

-0.14 

School Satisfaction (5-pt) 

No 

-0.08 

-0.05 

-0.06 

-0.03 

Cohort 3 (7th Graders) 

School Technology 

Classroom Activities (5-pt) 

Y ^ ^ 

0.37 

0.43 

0.15 

0.21 

Small-Group Work (5-pt) 

Yes** 

0.11 

0.14 

-0.04 

0.00 

Technical Problems (5-pt) 

Y gc 5 ! 5 ^ H* 

0.39 

0.39 

0.15 

0.15 

Student Self-Perceptions 

Technology Proficiency (5-pt) 

Y gc 5 ^ ^ H* 

0.43 

0.42 

0.27 

0.26 

School Satisfaction (5-pt) 

No 

-0.07 

-0.04 

-0.09 

-0.06 


Source: Student surveys completed during the 2005-06, 2006-07, and 2007-08 school years. 


Note, p < . 10. *p < .05. **p < .01. ***p < .001 . Items measured on either a 5-point or 7-point scale. 
■“For Cohort 3, Classroom Activities, the immersion effect is also net of initial status. 


Summary results show that Technology Immersion had positive effects on students in a number of 
areas. After controls for school poverty (percentage of economically disadvantaged students) and 
student economic disadvantage (qualification for free- or reduced-price lunch), estimated mean yearly 
rates of change for advantaged and disadvantaged immersion students revealed statistically significant 
positive growth trends favoring immersion students for Classroom Activities, Small-Group Work, and 
Technology Proficiency. Growth rates also showed that immersion students, compared to control, 
reported more Technical Problems using computers, with the growth-rate difference between groups 
statistically significant for both seventh and eighth graders. 

The Technology Immersion model also assumes that having daily access to and personal responsibility 
for laptop computers will allow immersion students to become more Self-Directed Learners and will 
increase their satisfaction with schoolwork (School Satisfaction). Contrary to expectations, as students 
in both the treatment and control groups advanced from sixth to higher grades, they reported being less 
self-directed learners and expressed less satisfaction with school. There were no statistically 
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significant differences between the views of immersion and control-group students. Sections to follow 
provide additional details for the HLM analyses. 

School Technology 

Table 5.2 provides statistics for the HLM growth models estimating the immersion effects on Cohorts 
2 and 3 students’ technology experiences. Specific scales are discussed below. 

Classroom Activities. Students reported the frequency with which their teachers had them use 
specific technology applications (e.g., use a word processor for writing, use a spreadsheet to calculate 
or graph, create a presentation) in their English language aits, mathematics, social studies, and science 
classes combined. Students reported their technology use on a 5-point scale ranging from 1 (never) to 
5 ( almost daily). As anticipated given the greater availability of hardware and software in immersion 
schools, treatment students had a significantly steeper growth rate for their frequency of technology 
use in core-subject classes. 


Table 5.2. Cohorts 2 and 3: Immersion (Fixed) Effect Analyses of School Technology Variables 


Dependent variable 
and predictor 

Classroom Activities 
(with technology) 

Small-Group Work 

Technical Problems 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Cohort 2 (8th Graders) 

Initial status (fall 2005) 

2.058 

39.61*** 

2.762 

55.26*** 

2.209 

50.50*** 

Immersion'* 

0.247 

3.00** 

-0.024 

-0.34 

-0.284 

-4 32*** 

School Poverty 

0.006 

2.64* 

0.002 

1.28 

0.005 

2.31* 

Economic Disadvantage 

-0.005 

-0.12 

0.002 

0.04 

-0.058 

-2.07* 

Growth rate 

0.099 

3.41** 

0.007 

0.43 

0.145 

5.15*** 

Immersion 

0.092 

2.26* 

0.091 

3.12** 

0.198 

5.57*** 

School Poverty 

-0.003 

-2.35* 

-0.001 

-1.15 

-0.002 

-2.14* 

Economic Disadvantage 

0.029 

1.8? 1 

0.003 

0.20 

0.019 

1.33 

Cohort 3 (7th Graders) 

Initial status (fall 2006) 

1.967 

22.45*** 

2.785 

41.57*** 

2.136 

37.66*** 

Immersion b,c 

0.436 

3.67** 

-0.015 

-0.18 

-0.139 

- 1.92 f 

School Poverty 

0.004 

1.13 

0.002 

0.79 

-0.003 

-1.67 

Economic Disadvantage 

0.002 

0.03 

-0.031 

-0.58 

-0.047 

-1.11 

Growth rate b 

1.067 

g 47*** 

-0.039 

-1.17 

0.152 

4 ^2*** 

Immersion 1 * 

0.217 

4 20*** 

0.144 

3.09** 

0.235 

4 92*** 

School Poverty 1 * 

-0.003 

0.044* 

-0.001 

-0.94 

0.001 

0.39 

Economic Disadvantage 

0.059 

1.58 

0.036 

1.25 

-0.001 

-0.05 

Initial status 1 * 

-0.464 

-7.49*** 

- 

- 

- 

- 


V < .10. *p < .05. **p < .01. ***p < .001. 

a Cohort 2 immersion students initially had significantly higher classroom activities scores and significantly lower 
technical problem scores. Separate latent variable regressions controlling for the effects of initial differences 
indicated that, in both cases, immersion was a significant predictor of the growth rate with and without controlling 
for initial differences. Thus, the coefficients from original growth models are reported here, 
b Cohort 3 immersion students initially had significantly higher classroom activities scores. A latent variable 
regression, controlling for the effects of the initial difference on the growth rate, revealed a significant immersion 
effect after controlling for initial differences. Thus, coefficients from the latent variable regression model are 
reported here. 

c Cohort 3 immersion students initially had significantly lower technical problem scores. A latent variable 
regression, controlling for the effect of the initial difference on the growth rate, indicated that immersion was a 
significant predictor of growth with and without controlling for initial differences. Thus, the coefficients from the 
original growth model are reported here. 
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For Cohort 2 students, the yearly rates of change in Classroom Activities involving technology were 
0.19 and 0.22 scale-score points for economically advantaged and disadvantaged immersion students, 
respectively. In contrast, advantaged and disadvantaged control students had flatter rates of change 
(0.10 and 0.13 scale-score points, respectively). For Cohort 3, the yearly rates of change in Classroom 
Activities for economically advantaged and disadvantaged immersion students were 0.37 and 0.43 
scale-score points, respectively, whereas advantaged and disadvantaged control students had much 
flatter rates of changes (0.15 and 0.21 scale-score points, respectively). Figure 5.1 compares the mean 
growth trajectories for the frequency of Classroom Activities for Cohorts 2 and 3 student groups. 




Figure 5.1. Estimated mean growth trajectories for the frequency of Classroom Activities for 
Cohorts 2 and 3 students, by economically advantaged and disadvantaged student groups in 
immersion and control schools. Estimated scale scores are displayed for disadvantaged students. 

Cohort 2 economically advantaged and disadvantaged students in immersion schools had spring 2008 
estimated Classroom Activities mean scores of 2.9 and 3.0, respectively, on the 5-point frequency 
scale, whereas mean scores for their control-group counterparts were 2.4 for both advantaged and 
disadvantaged students. Thus, despite significant increases in technology use by immersion students, 
mean use statistics indicated that students used various technology applications infrequently in 
classrooms (about once or twice a month). The growth trajectory for Cohort 3 immersion students was 
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notably steeper, with students in spring 2008 using technology applications from once or twice a 
month to once or twice a week (M =3.1 for advantaged and 3.3 for disadvantaged students). Across 
cohorts, students in control schools typically used technology applications just a few times a year. 

Small-Group Work. We also asked students to rate the frequency of their small-group interactions 
with classmates. Students rated statements, such as “we tutor or coach each other,” “brainstorm 
solutions to problems,” and “discuss assignments” on a 5-point scale ranging from 1 (never) to 5 
(i almost daily). Growth rate coefficients showed that students in immersion schools reported increasing 
opportunities for small-group work with their peers. Across cohorts, economically advantaged and 
disadvantaged immersion students had significantly positive yearly growth rates (0.10 and 0.10 scale- 
score points, respectively, for Cohort 2; 0.1 1 and 0.14 scale-score points, respectively, for Cohort 3). 
Quite the opposite, students at control campuses reported stable or less frequent small-group activities 
across survey times (yearly growth rates for advantaged and disadvantaged students ranged from 0.01 
to -0.04 scale-score points). 

Technical Problems. Given the increased availability of technology in immersion schools and 
classrooms, we reasoned that students might encounter more technical problems. Thus, we asked 
students to indicate on a 5-point scale about how often various Technical Problems happened when 
they tried to use a computer at school. Across Cohorts 2 and 3, growth rates showed that immersion 
students reported more technical problems using computers compared to control students. Figure 5.2 
shows that Cohort 2 immersion students initially reported fewer technical problems than control 
students, but by the end of eighth grade, both economically advantaged and disadvantaged immersion 
students reported more technical troubles. Still, mean scores in spring 2008 indicated that eighth 
graders, on average, rarely (a few times a year) or just sometimes (once or twice a month) had 
problems using computers at school. 



— o — Immersion: Advantaged Student — * — Immersion: Disadvantaged Student 

— -k — Control: Advantaged Student — -X — Control: Disadvantaged Student 


Figure 5.2. Estimated mean growth trajectories for the frequency of Technical Problems for 
Cohort 2 students, by economically advantaged and disadvantaged student groups in immersion 
and control schools. 


50 


Student Self-Perceptions 


Table 5.3 provides statistical details for the HLM growth models gauging students’ self-perceptions of 
their Technology Proficiency, Self-Directed Learning, and School Satisfaction. Individual scales are 
discussed below. 

Table 5.3. Cohorts 2 and 3: Immersion (Fixed) Effect Analyses for Student Self-Perception Variables 



Technology Proficiency 

Self-Directed Learning 

School Satisfaction 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Cohort 2 (8th Graders) 

Initial status (fall 2005) 

2.982 

54.45*** 

4.721 

107.37*** 

3.829 

130.76*** 

Immersion 

-0.002 

-0.02 

-0.043 

-0.78 

0.038 

1.03 

School Poverty 

0.002 

0.93 

0.003 

1.58 

0.000 

-0.36 

Economic Disadvantage 

-0.264 

-5.90*** 

-0.096 

-3.64** 

-0.140 

-6.31*** 

Growth rate 

0.256 

11.34*** 

-0.142 

-11.05*** 

-0.058 

_4 90*** 

Immersion 

0.094 

3.06** 

0.013 

0.74 

-0.018 

-0.99 

School Poverty 

-0.001 

-0.59 

0.000 

0.30 

0.001 

2.56* 

Economic Disadvantage 

0.004 

0.25 

0.002 

0.21 

0.029 

2.71** 

Cohort 3 (7th Graders) 

Initial status (fall 2006) 

2.951 

52.10*** 

— 

- 

3.797 

138.60*** 

Immersion 

-0.019 

-0.22 

- 

- 

0.050 

1.33 

School Poverty 

0.001 

0.45 

- 

- 

-0.001 

-1.05 

Economic Disadvantage 

-0.145 

-3.38** 

- 

- 

-0.084 

-3.81*** 

Growth rate 

0.266 

13.75*** 

- 

- 

-0.092 

-4.29*** 

Immersion 

0.168 

4.68*** 

- 

- 

0.022 

0.69 

School Poverty 

-0.001 

-1.37 

- 

- 

0.001 

0.95 

Economic Disadvantage 

-0.010 

-0.57 

- 

- 

0.035 

1.74 


f < .10. *p < .05. **p < .01. ***p < .001. 


Technology Proficiency. As a measure of Technology Proficiency, students rated their skills in using 
technology applications on a 5-point scale ranging from 1 (/ can do this not at all or barely ) to 5 (/ can 
do this extremely well). Students indicated their skill level on statements aligned with the Texas 
Technology Applications Standards. Results for Cohorts 2 and 3 replicated results from previous 
project years. Both economically advantaged and disadvantaged immersion students grew in 
Technology Proficiency at a significantly faster rate than their counterparts in control schools. 
Specifically, for Cohort 2 students, the yearly rate of change in Technology Proficiency for both 
economically advantaged and disadvantaged immersion students was 0.35 scale-score points. In 
contrast, both advantaged and disadvantaged control students had somewhat flatter rates of change for 
Technology Proficiency (0.26 scale-score points). The yearly rates of change in Technology 
Proficiency for economically advantaged and disadvantaged Cohort 3 immersion students were 0.43 
and 0.42 scale-score points, respectively, whereas advantaged and disadvantaged control students had 
slower rates of change (0.27 and 0.26 scale-score points, respectively). 

Self-Directed Learning. Self-direction, as measured by the SLI for this study, includes statements 
relative to students’ forethought (e.g., goal setting, strategic planning, self-efficacy beliefs, intrinsic 
effort), performance/volition control (e.g., attention focusing, self-monitoring, and help seeking), and 
self-reflection (e.g., self-evaluation, adaptivity). Although prior research suggested that the 
individualized learning opportunities allowed through one-to-one technology would positively affect 
students’ self-regulated learning, our results, consistent with previous years, revealed no significant 
immersion effects on Cohort 2 students’ growth in self-direction. As both immersion and control 
students progressed through eighth grade, their responses to statements revealed significantly negative 
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growth trends. For Cohort 2, the estimated yearly rates of change in self-direction for both advantaged 
and disadvantaged students in immersion schools was -0.13 scale-score points, compared to -0.14 
scale-score points, for their control-group counterparts. Overall findings indicated that students did not 
consider themselves to be strongly self-directed learners. 

School Satisfaction. Students also rated their level of School Satisfaction by indicating the extent of 
their agreement with statements on a 5-point scale ranging from 1 ( strongly disagree ) to 5 ( strongly 
agree). For example, students responded to items measuring their satisfaction with class work, the 
meaningfulness of class work, and the extent to which they perceived their class work to be useful to 
them in the future. As sixth graders, both immersion and control students generally agreed with 
statements measuring their school satisfaction. Flowever, both treatment- and control-group students 
reported lower levels of school satisfaction across time. The estimated yearly rates of change in 
satisfaction for Cohorts 2 and 3 immersion students ranged from -0.04 to -0.08 scale-score points. 
Similarly, control students expressed declining levels of satisfaction with their schoolwork (-0.03 to 
-.09 scale-score points per year). 

Immersion Effects on Student Engagement 

Greater technology access and use, we theorized, would cause improvements in student conduct, and 
consequently, fewer discipline problems and increased school attendance. Findings presented below 
show positive effects of Technology Immersion on student discipline and behavior but not on school 
attendance. 

Student Discipline and Behavior 

As one indicator of engagement, we collected student-level data from schools on disciplinary actions 
occurring during the 2007-08 school year. Texas requires that schools report each disciplinary action 
that results in a removal of a student from their regular academic program for a full school day, so we 
compared the frequency of the disciplinary occurrences at treatment and control schools for Cohorts 2 
and 3 students. The distributions of the disciplinary actions for students in each cohort were generally 
non-normal and negatively skewed. Flowever, because of the robustness of Mests of differences 
between mean scores to violations of the normality assumption (see Rasch & Guiard, 2004), this 
parametric procedure was used to examine differences between groups. Results show statistically 
significant differences between the frequency of student disciplinary actions at immersion and control 
schools, favoring immersion across two cohorts (Table 5.4). Figure 5.3 compares the average number 
of disciplinary actions for immersion and control schools for each student cohort. 


Table 5.4. Differences between Mean Number of Disciplinary Actions per Student at 
Treatment and Control Schools by Cohort 


Immersion 

Control 


Cohen's 

N Mean SI) 

N Mean SI) 

r- value 

d 

Cohort 2 (8th Graders) 2,624 0.54 1.59 

2,797 0.76 2.61 

3 73*** 

-0.11 

Cohort 3 (7th Graders) 2,590 0.45 1.44 

2,825 0.71 2.33 

5 00*** 

-0.13 


***p < .001. Note. Independent samples r-test for differences between average disciplinary actions per student at 
treatment and control schools. N = number of students. 


First, Cohort 2 eighth graders at immersion schools had significantly fewer disciplinary actions than 
control students (t = 3.73, p < 0.001). Specifically, 2,797 control-group students had an average of 
0.76 disciplinary actions compared to 2,624 immersion students who had an average of 0.54 
disciplinary events. Similarly, Cohort 3 seventh graders at immersion schools had significantly fewer 
disciplinary actions than students at control schools (t = 5.00, p < 0.001). In particular, 2,825 control- 
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group students had an average of 0.7 1 disciplinary actions compared to 2,590 immersion students who 
had an average of 0.45 disciplinary actions. Effect sizes for the mean differences between groups were 
small across cohorts (-0.11 and -0.13, respectively). 



H Immersion □ Control 


Figure 5.3. Average number of disciplinary actions per student for Cohorts 2 and 3. 

Overall, fourth-year findings on student discipline and behavior mirror results for the first through 
third project years. Evidence shows that students attending Technology Immersion schools have fewer 
disciplinary referrals than their counterparts in control schools. Although the estimated size of 
differences between groups is considered statistically small, having fewer disciplinary actions per 
student in middle schools may have practically important benefits. 

Student Attendance 

School attendance rates (absolute values). Another indicator of engagement is students’ school 
attendance. Accordingly, we compared the annual attendance rates for Cohort 2 students for the year 
before project implementation and for three implementation years, and Cohort 3 students for the year 
before implementation and for two implementation years (Table 5.5). 

Table 5.5. School Attendance Rates for Cohorts 2 and 3 Students 



Immersion 

Control 



Mean 

SD 

Mean 

SD 

Difference 

Cohort 2 (8th) 






2004-05 

97.26 

3.17 

97.41 

3.12 

-0.15 

2005-06 

96.73 

3.42 

97.11 

3.02 

-0.38 

2006-07 

96.20 

4.23 

96.76 

3.68 

-0.56 

2007-08 

95.88 

4.60 

96.13 

4.74 

-0.25 

Cohort 3 (7th) 




2005-06 

97.02 

3.34 

97.28 

3.08 

-0.26 

2006-07 

96.30 

4.14 

96.93 

3.63 

-0.63 

2007-08 

96.08 

4.20 

96.34 

4.17 

-0.26 


Results for Cohort 2 students show that the average attendance rate of immersion students was about 
0.2 percentage point lower than the attendance rate of control students in the year before 
implementation, and the attendance-rate gap increased incrementally to about 0.6 percentage point 
lower after two implementation years. However, the gap decreased to about 0.3 percentage point lower 
after three years of implementation. In the same way, the average attendance rate of Cohort 3 
immersion students was about 0.3 percentage point lower than the control group prior to project 
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implementation, and after one year, the attendance rate of immersion students was 0.6 percentage 
point lower. However, after two years, the gap had decreased to 0.3 percentage point. 

HLM analyses of attendance. To test the effects of immersion on student attendance, while 
controlling for school and student characteristics, we conducted HLM analyses. We used three-level 
HLM growth models to examine changes in school attendance rates over time. Table 5.6 presents the 
HLM statistics for each of the student cohorts. Results show that the effect of Technology Immersion 
on student attendance was negative but not by a statistically significant degree. 

Table 5.6. Cohorts 2 and 3: Immersion (Fixed) Effect Analyses of Student Attendance 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

t 

Cohort 2 (8th Graders) 

3-Level HLM Model 

Initial attendance (2005) 

97.587 

677.01*** 


Immersion 

-0.232 

-1.34 


School poverty 

0.014 

2.59* 


Disadvantaged 

-0.356 

-2.86** 


Growth rate 

-0.189 

-3.15*** 


Immersion 

-0.059 

-0.86 


School poverty 

0.00 1_ 

0.40 


Disadvantaged 

-0.288 

-8.35*** 

Cohort 3 (7th Graders) 

3-Level HLM Model 

Initial attendance (2006) 

97.600 

711.39*** 


Immersion 11 

-0.476 

-2.01* 


School poverty 

0.022 

4 12*** 


Disadvantaged 

-0.567 

-4 3i*** 


Growth rate 

-0.295 

-3.15** 


Immersion 1 * 

-0.100 

-0.85 


School poverty 

-0.001 

-0.32 


Disadvantaged 

-0.115 

-2.40* 


*p < .05. **p < .01. ***p < .001. 

'‘Immersed students in Cohort 3 had significantly lower initial 2006 attendance rates. A latent 
variable regression was run to control for the effect of this initial difference on the growth rate. 

The immersion effect was not a significant predictor of the growth rate with and without 
controlling for initial differences. Thus, the coefficients from the original growth model are 
reported here. 

Average school attendance rates for economically advantaged Cohort 2 immersion and control-group 
students in schools with average rates of school poverty decreased as students advanced from fifth to 
eighth grade. The yearly estimated negative rate of change in attendance for immersion students (-0.25 
percentage point) was greater than the annual change for control students (-0.19 percentage point). 
Thus, at the end of eighth grade, advantaged students in immersion schools had an estimated average 
attendance rate of 96.6% percent compared to 97.0% for control students (see Figure 5.4). Attendance 
rates for economically disadvantaged students decreased at a faster pace, with yearly negative change 
rates for disadvantaged students in immersion schools greater than the rates for control students (-0.54 
percentage point versus -0.48 point, respectively). Thus, by the end of eighth grade, economically 
disadvantaged students in immersion schools had an attendance rate of 95.4% compared to 95.8% for 
control students. Similar results were found for Cohort 3 (Figure 5.5). While comparable trends were 
found across both cohorts, the differences in the growth rates were not statistically significant. 
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Figure 5.4. Estimated attendance rates for Cohort 2 economically advantaged and 
disadvantaged students in immersion and control schools with average rates of school poverty. 
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Figure 5.5. Estimated attendance rates for Cohort 3 economically advantaged and 
disadvantaged students in immersion and control schools with average rates of school poverty. 

Conclusions 

In the fourth project year, we investigated the effects of Technology Immersion on the learning 
experiences and competencies of students in Cohort 2 (eighth graders) and Cohort 3 (seventh graders). 
After controlling for important school and student characteristics, results across cohorts, consistent 
with previous evaluation years, confirmed the hypothesized positive effects of immersion on some 
mediating variables, but the outcomes for other variables were contrary to expectations. Key findings 
include the following. 
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Across cohorts, Technology Immersion positively affected students’ classroom technology use 
(Classroom Activities) and interactions with peers (Small-Group Work). Students in 
immersion schools used various technology applications significantly more often in their core- 
subject classrooms than control students. They also had significantly more frequent 
opportunities to learn in small groups with their classmates. 

Across cohorts, Technology Immersion positively affected students’ Technology Proficiency, 
Cohorts 2 and 3 immersion students grew in proficiency at a significantly faster rate than 
control students. Moreover, Technology Immersion substantially narrowed the technology 
proficiency gap between economically advantaged and disadvantaged students. Economically 
disadvantaged immersion students had higher proficiency levels in spring 2008 than 
advantaged control students. 

Across cohorts, Technology Immersion students, compared to control-group students, reported 
significantly more Technical Problems using computers over time. 

Across cohorts, immersion and control-group students expressed similar levels of school 
satisfaction, with both groups reporting significantly lower levels of school satisfaction as they 
progressed to higher grade levels. 

Technology Immersion students in Cohort 2, who had access to personal laptop computers and 
resources for learning, regarded themselves as no more Self-Directed Learners than control 
students. As both immersion and control students progressed from sixth to eighth grade they 
reported less self-directed learning behaviors. 

Across cohorts, Technology Immersion positively affected student discipline and behavior. 
Students in immersion schools, on average, had proportionately fewer behavioral and 
disciplinary problems that removed them from the regular academic program than their 
counterparts in control schools. 

Contrary to expectations, results indicated that Technology Immersion had a negative but non- 
significant effect on students’ school attendance. Across two cohorts, estimated attendance 
rates for immersion students were slightly lower than attendance rates for control-group 
students. 



6. Effects of Technology Immersion on Student Achievement 


The overarching goal of the Texas Technology Immersion project is increasing middle school 
students’ achievement in core academic subjects (English language aits, mathematics, science, and 
social studies) as measured by the Texas Assessment of Knowledge and Skills (TAKS). We theorized 
that students who attended fully immersed schools would experience school and classroom conditions 
that promoted more individualized learning, more intellectually challenging work, and stronger 
engagement in school and learning. In turn, changes in students and their learning experiences would 
contribute to enhanced performance on state assessments. 

In the fourth year of the project, as detailed in previous report chapters, we have noted teachers’ 
substantial growth across years in the frequency and quality of classroom technology use, as well as 
improvements in students’ technology proficiency and frequency of technology use. The following 
sections present academic achievement results for Cohort 2 students (eighth graders) and Cohort 3 
students (seventh graders) who were enrolled continuously in the 21 Technology Immersion and 21 
control schools through TAKS testing in April 2008. Additionally, we provide information on the 
progress of Cohort 1 students (ninth graders) who attended Technology Immersion and control schools 
from sixth through eighth grade and then were enrolled through 2008 TAKS testing in high schools 
that typically did not provide individual laptops for students. As far as we know, only one small high 
school provided one-to-one student access to laptops. Ninth graders at high schools, however, almost 
certainly continued to have access to computers in computer labs, in classrooms, or at home. 

Texas Assessment of Knowledge and Skills 
Passing Standards and Scale Scores 

The TAKS is Texas’ criterion-referenced assessment that measures students’ mastery of the state’s 
content standards, the Texas Essential Knowledge and Skills (TEKS). At the middle school, TAKS 
assesses reading and mathematics at grades 6, 7, and 8, writing at grade 7, and science and social 
studies at grade 8. The TAKS also assesses reading and mathematics at grade 9. This study uses 
several types of TAKS scores. 

• Met the standard. This score represents satisfactory academic achievement. Students who 
met this standard performed at a level that was at or somewhat above the state passing 
standard. Thus, students demonstrated a sufficient understanding of the knowledge and skills 
measured at the grade level. 

• Commended performance. This score represents high academic achievement. Students who 
met this standard performed at a level that was considerably above the state passing standard. 
Therefore, students demonstrated a thorough understanding of the knowledge and skills 
measured at the grade level. 

• TAKS scale score. The scale score is a statistic that provides a comparison of scores with a 
standard set at 2100 for each grade level. The scale score can be used to determine whether a 
student met the minimum standard or achieved commended performance, but it cannot be 
used to evaluate a student’s progress across grades or subject areas. TAKS scale scores are 
used to calculate standardized scores for this study. 
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Texas has phased-in increasingly rigorous passing standards on the TAKS. In 2004-05, passing 
standards recommended for reading, mathematics, writing, social studies, and grade 5 science by the 
State Board of Education panel were fully implemented. For the newer grade 8 science test, the panel- 
recommended standard had to be met in 2007-08. For this study, all TAKS scores reported are based 
on panel-recommended standards. 

Standard Scores 

In addition to the scores provided by the Texas Education Agency (TEA), researchers generated 
standard scores that were used to compare student progress on TAKS across grade levels. A 
standardized score — or z score — was calculated for each student and for every testing occasion and 
subject. The z score is calculated by subtracting the statewide mean grade-level scale score from each 
student’s scale score and dividing by the statewide scale score standard deviation. The z score, which 
has a mean of zero and a standard deviation of 1 .0, indicates how many standard deviations from the 
mean a score lies. One characteristic of z scores is that about half of the scores are negative, and 
negative scores may be difficult to fully understand. To overcome this limitation, we have transformed 
students’ z scores into normalized scores, or T scores. T scores are scores with a mean of 50 and a 
standard deviation of 10. Thus, a student who scores at the state average will have a TAKS T score of 
50. A student who has a score of 60 will be one standard deviation above the state average, and a 
student who has a score of 40 will be one standard deviation below the state average. 

Progress in Meeting TAKS Standards 


TAKS Reading 

Students’ progress in meeting TAKS passing and commended performance standards is one measure 
of student academic outcomes. Information in Table 6.1 compares the absolute performance of 
students in immersion and control schools for TAKS reading (Cohorts 2 and 3) and also compares the 
reading performance of post-immersion and control ninth graders (Cohort 1). 


Table 6.1. TAKS Passing and Commended Performance Rates for Reading 


Cohort 

Group 

N 

2004 

Percent 

2005 

Percent 

2006 

Percent 

2007 

Percent 

2008 

Percent 

Baseline to 
2008 

Difference 

Met Standard 

Cohort 1 

Grade 5 to 9 

Post-immersion 

1,332 

68.5 

77.0 

73.8 

86.9 

87.8 

19.3 

Control 

1,552 

74.5 

83.0 

79.1 

88.7 

90.2 

15.7 

Cohort 2 

Grade 5 to 8 

Immersion 

1,434 

— 

68.1 

88.6 

82.3 

93.5 

25.4 

Control 

1,569 

— 

74.7 

91.7 

85.0 

95.4 

20.7 

Cohort 3 

Grade 5 to 7 

Immersion 

1,548 

— 

— 

74.4 

87.5 

83.5 

9.1 

Control 

1,771 

— 

— 

81.0 

92.4 

86.9 

5.9 

Commended 

Performance 

Cohort 1 

Grade 5 to 9 

Post-immersion 

1,332 

19.9 

29.3 

17.1 

35.5 

32.1 

12.2 

Control 

1,552 

23.5 

35.7 

18.9 

41.0 

33.6 

10.1 

Cohort 2 

Grade 5 to 8 

Immersion 

1,434 

— 

18.1 

30.5 

22.0 

48.5 

30.4 

Control 

1,569 

— 

19.4 

34.6 

21.0 

50.0 

30.6 

Cohort 3 

Grade 5 to 7 

Immersion 

1,548 

— 

— 

19.1 

40.8 

26.6 

7.5 

Control 

1,771 

— 

— 

21.1 

47.7 

27.6 

6.5 


Source: Analysis of individual student data from TEA. 

Note. Cohorts 2 and 3 include students in 21 treatment and 21 control schools that had TAKS scores and attended the same school 
across years. Cohort 1 (post-immersion) includes treatment and control students that attended various high schools as ninth 
graders during the fourth year. Italic numbers denote baseline scores. Bold numbers denote superior baseline-to-2008 differences. 
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Results show that Cohorts 2 and 3 students at Technology Immersion schools had slightly lower 
passing rates in spring 2008 for TAKS reading than students at control campuses. However, students 
at immersion campuses had greater baseline-to-2008 passing increases. For Cohort 2, TAKS-score 
comparisons between 2005 (5th grade baseline) and 2008 (8th grade) revealed larger reading gains for 
the immersion group (25.4 percentage points versus 20.7 points for the control group). Similarly, for 
Cohort 3, the TAKS passing rate difference between 2006 (5th grade baseline) and 2008 (7th grade) 
favored students at immersion schools (9.1 percentage points versus 5.9 points for control students). 
Similarly, for Cohort 1 students who left treatment and control schools and enrolled in high schools, 
TAKS-score comparisons between 2004 (5th grade baseline) and 2008 (9th grade) revealed larger 
reading gains for the immersion group (19.3 percentage points versus 15.7 points for the control 
group). 

For commended performance, Cohorts 1, 2, and 3 students at control schools had slightly higher 2008 
achievement rates. Cohorts 1 and 3 students at immersion schools had slightly larger baseline-to-2008 
gains, whereas Cohort 2 students at immersion and control schools had nearly identical gains. 

TAKS Mathematics 

Results for TAKS mathematics show that Cohort 2 students at immersion schools had higher 
mathematics passing rates in spring 2008 than students at control campuses. They also had 
considerably larger baseline-to-2008 gains (4.8 percentage points versus -5.9 percentage points). 
Cohort 3 students in immersion schools and Cohort 1 post-immersion ninth graders had slightly lower 
TAKS mathematics passing rates in 2008 than students at control campuses; however, TAKS-score 
comparisons across years revealed smaller TAKS mathematics passing-rate decreases for the 
immersion and post-immersion students (Table 6.2). 


Table 6.2. TAKS Passing and Commended Performance Rates for Mathematics 


Cohort 

Group 

N 

2004 

Percent 

2005 

Percent 

2006 

Percent 

2007 

Percent 

2008 

Percent 

Baseline to 
2008 

Difference 

Met Standard 

Cohort 1 

Grade 5 to 9 

Post-immersion 

1,345 

70.9 

62.9 

66.3 

71.4 

66.8 

-4.1 

Control 

1,560 

73.5 

68.2 

68.6 

71.7 

67.1 

-6.4 

Cohort 2 

Grade 5 to 8 

Immersion 

1,435 

— 

74.4 

73.7 

74.1 

79.2 

4.8 

Control 

1,576 

— 

80.3 

76.0 

75.0 

74.4 

-5.9 

Cohort 3 

Grade 5 to 7 

Immersion 

1,555 

— 

— 

79.3 

74.9 

75.2 

-4.1 

Control 

1,809 

— 

— 

85.0 

79.0 

77.4 

-7.6 

Commended 

Performance 

Cohort 1 

Grade 5 to 9 

Post-immersion 

1,345 

24.3 

19.6 

11.2 

15.8 

20.5 

-3.8 

Control 

1,560 

24.8 

22.0 

9.7 

13.8 

20.9 

-3.9 

Cohort 2 

Grade 5 to 8 

Immersion 

1,435 

— 

23.6 

23.6 

16.9 

21.1 

-2.5 

Control 

1,576 

— 

25.9 

23.6 

12.7 

16.0 

-9.9 

Cohort 3 

Grade 5 to 7 

Immersion 

1,555 

— 

— 

32.9 

29.5 

16.8 

-16.1 

Control 

1,809 

— 

— 

37.7 

28.2 

15.6 

-22.1 


Source: Analysis of individual student data from TEA. 

Note. Cohorts 2 and 3 include students in 21 treatment and 21 control schools that had TAKS scores and attended the same school 
across years. Cohort 1 (post-immersion) includes treatment and control students that attended various high schools as ninth 
graders during the fourth year. Italic numbers denote baseline scores. Bold numbers denote superior baseline-to-2008 differences. 

Overall, students had greater difficulty meeting commended standards for mathematics compared to 
reading. For Cohorts 2 and 3, students at immersion schools had slightly higher 2008 commended 
performance rates, and immersion students had smaller baseline-to-2008 rate declines (-2.5 percentage 
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points versus -9.9 points for Cohort 2; -16.1 percentage points versus -22.1 points for Cohort 3). For 
Cohort 1, post-immersion and control ninth graders had nearly identical 2008 commended 
performance rates and declines over time. 

TAKS Social Studies, Science, and Writing 

The TAKS reading and mathematics tests are administered annually, whereas TAKS tests are 
administered periodically in other subject areas. In the fourth year, Cohort 2 eighth graders completed 
TAKS social studies and science assessments, while Cohort 3 seventh graders completed TAKS 
writing. Baseline measures were available for TAKS science in grade 5 and writing in grade 4. There 
was no pre-measure for TAKS social studies. 

Results for TAKS social studies in Table 6.3 show that Cohort 2 eighth graders at control schools had 
slightly higher TAKS passing rates and commended performance rates for social studies in 2008 
(89.1% versus 88.6%; 31.6% versus 29.9%). Cohort 2 students at immersion schools had lower TAKS 
passing rates for science in 2008 (61.4%) than control students (67.3%) but nearly identical baseline to 
2008 passing rate increases (5.0 and 4.9 percentage points, respectively). Similarly, Cohort 2 
immersion students had lower TAKS commended performance rates for science in 2008 (15.7%) than 
control students (18.0%) but nearly identical baseline-to-2008 passing rate decreases (-5.9 and -5.7 
percentage points, respectively). 

Table 6.3. Cohort 2 (Eighth Graders in 2007-08): 

TAKS Passing and Commended Performance Rates for Social Studies and Science 


TAKS Test 

Group 

N 

2005 
Grade 5 
Percent 

2008 
Grade 8 
Percent 

Baseline to 
2008 

Difference 

Met Standard 

Social 

Immersion 

1,546 

— 

88.6 

— 

Studies 

Control 

1,663 

- 

89.1 

- 

Science 

Immersion 

1,430 

56.4 

61.4 

5.0 


Control 

1,550 

62.4 

67.3 

4.9 

Commended 1 

Performance 

Social 

Immersion 

1,546 

— 

29.9 

— 

Studies 

Control 

1,663 

- 

31.6 

- 

Science 

Immersion 

1,430 

21.6 

15.7 

-5.9 


Control 

1,550 

23.7 

18.0 

-5.7 


Source: Analysis of individual student data from TEA. 

Notes. Students had TAKS scores and attended the same school across years. Italic 
numbers denote baseline scores. Bold numbers denote superior baseline-to-2008 
differences. 

Table 6.4 shows that Cohort 3 students at immersion and control schools had similar TAKS passing 
rates for writing in 2008, but immersion students’ TAKS-score gains between 2005 (4th grade 
baseline) and 2008 (7th grade) were larger. Flowever, control students achieved commended 
performance in writing at a higher rate and had larger gains than immersion students (14.0 percentage 
points versus 11.8 points). 
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Table 6.4. Cohort 3 (Seventh Graders in 2007-08): 

TAKS Passing and Commended Performance Rates for Writing 


TAKS Test 

Group 

N 

2005 
Grade 4 
Percent 

2008 
Grade 7 
Percent 

Baseline to 
2008 

Difference 

Met Standard I 

Writing 

Immersion 

1,435 

89.0 

93.9 

4.9 


Control 

1,641 

92.8 

94.1 

1.3 

Commended 

Performance 





Writing 

Immersion 

1,435 

19.0 

30.8 

11.8 


Control 

1,641 

21.4 

35.4 

14.0 


Source: Analysis of individual student data from TEA. 

Notes. Students had TAKS scores and attended the same school across years. Italic 
numbers denote baseline scores. Bold numbers denote superior baseline-to-2008 
difference. 

Altogether, TAKS passing rates provide important evidence that helps to understand student progress 
toward meeting state standards — however, additional statistical analyses are necessary to assess the 
effects of immersion on student achievement. 

Effects of Immersion on Academic Achievement 

Researchers used hierarchical linear modeling (HLM) to estimate the effects of immersion on 
students’ academic achievement. HLM is a “value added” methodology. That is, after controlling for 
students’ initial achievement and characteristics and accounting for variance at the student and school 
levels, researchers can assess the “value added” by the treatment. The analyses to follow contrast the 
achievement of three student cohorts: 

• Cohort 2, before and after three immersion years (sixth to eighth grade), 

• Cohort 3, before and after two immersion years (sixth to seventh grade), and 

• Cohort 1 , before and after three immersion years and one post-immersion year (sixth to ninth 

grade). 

Immersion effects for Cohort 2 arc estimated for TAKS reading, mathematics, social studies, and 
science T scores. For Cohort 3, effects are estimated for TAKS reading, mathematics, and writing 
T scores. The enduring effects of attending an immersion school are estimated for Cohort 1 students 
for TAKS reading and mathematics T scores. We used three-level HLM growth models to examine 
changes in students’ TAKS reading and mathematics achievement over time. For TAKS social studies, 
science, and writing, students had scores for only two time points, so data analysis involved two-level 
HLM models. (See Appendix E for technical detail on the HLM models.) 

The availability of longitudinal achievement data for three student cohorts allowed researchers to 
evaluate program effects by examining the importance of group differences, and the replicability or 
truth of group differences across cohorts and outcome measures (e.g., Cohen, 1994; Schmidt, 1996). 
Since small effects are noteworthy when evidence indicates that effects arc replicable, we have 
reported effects as statistically significant at less conservative levels {p < .10) when findings provided 
evidence of important trends. 
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TAKS Reading 

Cohorts 2 and 3 (Technology Immersion) 

TAKS reading achievement growth trajectories were estimated for Cohorts 2 and 3 students in 
immersion and control schools. Three-level HLM growth models examined the extent to which 
student achievement varied across time, students, and schools. Given the complexity of interpreting 
growth models, we constrained our final models to include school and student predictors that exhibited 
strong associations with achievement (i.e., school and student poverty). In the HLM growth model, 
Level 1 is a repeated-measures model (i.e., TAKS assessment time within students) that captures the 
key features of growth (e.g., initial status, rate of change). Time is the point at which students 
completed assessments each spring (Cohort 2, 0 = 2005, 1 = 2006, 2 = 2007, 3 = 2008; Cohort 2, 

0 - 2006, 1 = 2007, 2 = 2008). 

The between-students model (Level 2) modeled differences between students in features of growth 
(e.g., initial status, rate of change), after adjusting for students’ economic status (1 if economically 
disadvantaged [i.e., eligible for the federal free- and reduced-price lunch program], 0 if not). At the 
school level (Level 3), we examined how students’ initial status and growth varied across schools as a 
function of school-level random effects, as well as school conditions, including group membership 
(1 for immersion, 0 for the control group) and school poverty (percentage of economically 
disadvantaged students attending a school). School poverty rates ranged from 31% to 100%, with a 
mean of 68.5%. Thus, we hypothesized that being in an immersion school was positively related to 
students’ growth in achievement, after controlling for the poverty level of the school. 

Separate HLM growth models were used to determine the effects of immersion on Cohort 2 and 
Cohort 3 students’ growth in TAKS reading achievement (Table 6.5). Growth models estimated 
school mean rates of change for immersion and control students, as well as the separate effects of 
student economic disadvantage and the school poverty concentration on reading. Analyses for 
Cohort 2 involved 1,571 immersion and 1,697 control students. Comparison groups had nearly 
equivalent proportions of students included in longitudinal analyses (60.9% for immersion and 59.4% 
for control). Cohort 3 analyses involved 1,690 immersion and 1,965 control students. As with 
Cohort 1, analyses involved nearly equal proportions of students across groups (66.4% for immersion 
and 69.1% for control). 


Table 6.5. HLM Statistics for Cohort 2 and Cohort 3 Students: 
Effects of Immersion on TAKS Reading Achievement Growth Rates 



Cohort 2 (Eighth Graders) 
N = 3,268 

Cohort 3 (Seventh Graders) 
N = 3,655 

Dependent variable 
and predictor 

Gamma 

Coefficient 

1- value 

Gamma 

Coefficient 

1- value 

Initial mean status 
(2005/2006 TAKS T score) 

52.427 

102.75*** 

52.775 

99.01*** 

Immersion 

-0.315 

-0.54 

-0.751 

-1.31 

School poverty 

-0.091 

-6.69*** 

-0.083 

_4 

Economic disadvantage 

-5.664 

-9.50*** 

-5.123 

-6.83*** 

Growth rate 

0.131 

1.08 

-0.104 

-0.53 

Immersion 

0.234 

1.55 

0.105 

0.58 

School poverty 

0.010 

1.95^ 

0.015 

2.38* 

Economic disadvantage 

0.308 

2.95** 

0.109 

0.64 


'p < .10. *p < .05. **p < .01. ***p < .001. 
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As Table 6.5 shows, the initial mean TAKS reading status for the Cohort 2 reference group (an 
economically advantaged eighth grader in a control school with an average level of school poverty) is 
estimated at 52.43 (the mean 2005 TAKS reading T score). The coefficient representing immersion 
(-0.315) shows that students in immersion schools had lower initial TAKS reading T scores (52.1 1) 
than control students. Considering that differences among schools in students’ initial achievement may 
be related to subsequent rates of change, we used statistical tests to establish that those differences did 
not affect the estimations of student growth. Coefficients for initial status also showed that 
economically disadvantaged students and students attending schools with above average levels of 
poverty stalled behind their more advantaged counterparts in reading ability (-5.66 and -0.09 T-score 
points, respectively). 

After controlling for prior achievement and student and school levels of poverty, results show there 
was no statistically significant effect of immersion on Cohort 2 students’ growth rate for TAKS 
reading. Reading achievement for advantaged students in control schools (with average poverty) 
increased by 0.13 T’-scorc point per year. The coefficient for immersion (0.234) indicates that reading 
scores for advantaged students in immersion schools (with average poverty) increased at a slightly 
faster rate (0.37 T-score point per year) compared to control-group students (0.131 + 0.234 = 0.365). 
Economically disadvantaged eighth graders at both immersion and control schools grew in reading 
achievement at significantly faster rates than their more advantaged peers (0.67 T-score point per year 
for immersion and 0.44 T-score point for control students). Figure 6.1 illustrates the estimated mean 
TAKS reading growth trajectories for advantaged and disadvantaged Cohort 2 students by school 
comparison group. 



♦ Immersion: Advantaged Students — * — Immersion: Disadvantaged Students 

— -A — Control: Advantaged Students — — Control: Disadvantaged Students 


Figure 6.1. Estimated mean TAKS reading achievement growth trajectories for Cohort 2 
economically advantaged and disadvantaged student groups in immersion and control schools. 
Growth rate difference between the immersion and control groups is statistically insignificant. 

TAKS reading outcomes for Cohort 3 students, similarly, showed no statistically significant effect of 
immersion on seventh graders’ reading achievement. The estimated reading T scores of advantaged 
seventh graders in control schools with average poverty decreased (-0. 10 T-score point per year), 
while the scores for advantaged students in immersion schools remained stable (0.00 T-score point per 
year). Economically disadvantaged seventh graders at both immersion and control schools grew in 
reading at a slightly faster rate than their more advantaged counterparts. 
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Across both Cohorts 2 and 3, the extent of school poverty was a statistically significant positive 
predictor of students TAKS reading growth rate. With each percentage point increase in school 
poverty. Cohorts 2 and 3 students’ reading T score increased by 0.01 and 0.02 T-score point per year, 
respectively. 

Cohort 1 (Post-Immersion) 

We also estimated the continuing effect of attending an immersion school on Cohort 1 students’ 

TAKS reading T scores using an HLM growth model (Table 6.6). In the three-level HLM model, level 
1 is a repeated-measures model (i.e., TAKS assessment time within students) that captures the key 
features of growth. Time is the point at which students completed assessments each spring (0 = 2004, 

1 = 2005, 2 = 2006, 3 = 2007, 4 = 2008-post immersion). Level 2 (between-students model) modeled 
features of growth after adjusting for students economic status. At Level 3 (school level), we 
examined how students’ initial status and growth varied across schools as a function of school- level 
random effects, as well as school conditions, including group membership (1 for post-immersion, 0 for 
the control group) and school poverty. School poverty was a continuous variable depicting the 
concentration of economically disadvantaged students in the students’ feeder middle schools. Analyses 
involved 1,506 post immersion students and 1,805 control students, with similar proportions of 
students included in analyses (61.0% and 65.7%, respectively). 


Table 6.6. HLM Statistics for Cohort 1 Students: Enduring Effects 
of Immersion on TAKS Reading Achievement Growth Rates 


Dependent variable 
and predictor 

TAKS Reading 
A = 3,311 

Gamma 

Coefficient 

t-value 

Initial mean status 



(2004 TAKS T score) 

53.283 

81.430*** 

Immersion 

-1.205 

-1.650 

School poverty 

-0.072 

-4.451*** 

Economic disadvantage 

-6.271 

-10.707*** 

Growth rate 

-0.014 

-0.158 

Immersion 

0.203 

1.932 1 

School poverty 

0.008 

2.902** 

Economic disadvantage 

0.367 

4 371*** 


f p < .10. *p < .05. **p < .01. ***p < .001. 

Note. Cohort 1 includes treatment and control students that attended various 
high schools as ninth graders in the fourth year. 


TAKS reading outcomes for ninth graders reported in Table 6.6 show that after controlling for 
students’ prior reading achievement, the level of middle-school poverty, and students’ economic 
disadvantage, there was a statistically significant positive sustaining effect of immersion (p < .06) on 
the TAKS reading T scores of students who had attended immersion middle schools and then moved 
on to mainly traditional high schools. Reading achievement for advantaged control students (with 
average school poverty) decreased by 0.01 T- score point per year, whereas the reading achievement of 
advantaged post-immersion students increased by 0.19 T-score point per year. Economically 
disadvantaged students grew in reading achievement at significantly faster rates than their advantaged 
peers. Given the positive treatment-group boost, disadvantaged post-immersion students grew at a 
notably faster rate than their control-group counterparts (0.56 T-score point per year versus 0.35 
point). Figure 6.2 illustrates how the more positive TAKS reading growth trajectories of post- 
immersion students has narrowed the initial reading achievement gap between the treatment and 
control groups. 
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Figure 6.2. Estimated mean TAKS reading achievement growth trajectories for Cohort 1 
economically advantaged and disadvantaged post-immersion and control students. The growth 
rate difference between post-immersion and control groups is statistically significant (p < .10). 

TAKS Mathematics 

Cohorts 2 and 3 ( Technology Immersion) 

Similar to reading, we estimated the TAKS mathematics achievement growth trajectories for 
Cohorts 2 and 3 students in immersion and control schools (Table 6.7). Three-level HLM growth 
models were used to examine the extent to which mathematics achievement varied across time (the 
point at which students completed TAKS assessments each spring), students, and schools. 


Table 6.7. HLM Statistics for Cohort 2 and Cohort 3 Students: 

Effects of Immersion on TAKS Mathematics Achievement Growth Rates 



Cohort 2 (Eighth Graders) 
N = 3,268 

Cohort 3 (Seventh Graders) 
N = 3,655 

Dependent variable 
and predictor 

Gamma 

Coefficient 

Aval lie 

Gamma 

Coefficient 

Aval lie 

Initial mean status 
(2004/2005 TAKS T score) 

52.152 

gi 

52.557 

79.46*** 

Immersion 

-0.891 

-1.37 

-1.465 

-2.19* 

School poverty 

-0.052 

-2.70* 

-0.040 

-2.10* 

Economic disadvantage 

-4.623 

-7 78*** 

-3.893 

-5.90*** 

Growth rate 

-0.230 

-1.20 

-0.417 

-1.93^ 

Immersion 

0.653 

2.58* 

0.787 

2.69* 

School poverty 

0.005 

0.70 

0.025 

3.05** 

Economic disadvantage 

0.179 

1.61 

-0.325 

-1.64 


'p < .10. *p < .05. **p < .01. ***p < .001. 


Results for Cohort 2 students show that control students initially had an estimated mean mathematics 
T score of 52.15, whereas immersion students began with a lower estimated mathematics score 
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(51.25). Economically disadvantaged students and students attending schools with above average 
levels of poverty started significantly behind their more advantaged peers in math ability (-4.62 
T-score points and -0.05 point, respectively). After controlling for student and school levels of 
poverty. Technology Immersion had a positive and statistically significant effect on students’ growth 
rate for TAKS mathematics {p < .05). Estimated mathematics achievement for economically 
advantaged students in immersion schools (with average poverty) increased by about 0.42 T-score 
point per year (coefficient of 0.653), while the math scores of their control-group counterparts 
decreased by about 0.23 T-score point per year (coefficient of -0.230). Economically disadvantaged 
students in immersion schools grew in mathematics achievement at an even faster rate (about 0.60 T- 
score point per year) that well out-paced economically disadvantaged control students (-0.05 point per 
year). Figure 6.3 illustrates the estimated mean TAKS mathematics growth trajectories for Cohort 2 
advantaged and disadvantaged students at immersion and control schools. 



♦ Immersion: Advantaged Students — ■ — Immersion: Disadvantaged Students 

— -0 — Control: Advantaged Students — -S — Control: Disadvantaged Students 


Figure 6.3. Estimated mean TAKS mathematics achievement growth trajectories for Cohort 2 
economically advantaged and disadvantaged immersion and control students. The growth rate 
difference between the immersion and control groups is statistically significant (p < .05). 

Similarly, TAKS mathematics outcomes for Cohort 3 revealed a statistically significant positive effect 
of immersion on seventh graders’ math achievement for both advantaged and disadvantaged students 
( p < .05). The mathematics T scores of advantaged seventh graders in immersion schools (with 
average poverty) increased (0.37 T- score point per year), while the scores for advantaged students in 
control schools decreased (-0.42 T-score point per year). Similarly, the math scores for economically 
disadvantaged seventh graders at immersion schools increased (0.05 T-score point per year), whereas 
disadvantaged control-group students had a negative growth trend (-0.74 T-score point per year). 
Figure 6.4 illustrates the estimated mean TAKS mathematics growth trajectories for Cohort 2 
advantaged and disadvantaged students at immersion and control schools. 
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Figure 6.4. Estimated mean TAKS mathematics achievement growth trajectories for Cohort 3 
economically advantaged and disadvantaged immersion and control students. The growth rate 
difference between the immersion and control groups is statistically significant (p < .05). 

Cohort 1 (Post-Immersion) 

We also examined the enduring effect of attending an immersion school on Cohort 1 students’ TAKS 
mathematics T scores. Like reading, we used a three-level HLM growth model to estimate the TAKS 
mathematics growth trajectories for Cohort 1 post-immersion and control students (Table 6.8). 
Analyses involved 1,506 post-immersion ninth graders and 1,805 control ninth graders, with similar 
proportions of students included in analyses (61.0% and 65.7%, respectively). 


Table 6.8. HLM Statistics for Cohort 1 Students: Enduring Effects 
of Immersion on TAKS Mathematics Achievement Growth Rates 


Dependent variable 
and predictor 

TAKS Mathematics 
IV = 3,311 

Gamma 

Coefficient 

f-value 

Initial mean status 



(2004 TAKS T score) 

52.372 

71.820*** 

Immersion 

-1.122 

-1.270 

School poverty 

-0.046 

-2.307* 

Economic disadvantage 

-4.523 

-9.029*** 

Growth rate 

0.221 

1.726 t 

Immersion 

0.313 

1.602 

School poverty 

0.008 

1.398 

Economic disadvantage 

-0.126 

-1.073 


'p < .10. *p < .05. **p < .01. ***p < .001. 

Note. Cohort 1 includes treatment and control students that attended various 
high schools as ninth graders in the fourth year. 
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TAKS mathematics outcomes for Cohort 1 revealed that after controlling for student and school levels 
of poverty, there was no statistically significant effect of immersion on ninth graders’ growth rate for 
TAKS mathematics. The immersion effect was positive but not by a significant margin. The 
mathematics T-scores of advantaged post-immersion students increased by about 0.53 T’-scorc point 
per year compared to 0.22 T-score point for control-group students. Economically disadvantaged 
post-immersion ninth graders grew in mathematics achievement at a faster rate (0.41 T-score point per 
year) than advantaged control students (0.22 point per year) and disadvantaged control students (0.10 
point per year). 



♦ Post-Immersion: Advantaged Students — ■ — Post-Immersion: Disadvantaged Students 

— -0 — Control: Advantaged Students — -H — Control: Disadvantaged Students 


Figure 6.5. Estimated mean TAKS mathematics achievement growth trajectories for Cohort 1 
economically advantaged and disadvantaged post-immersion and control students. The growth rate 
difference between the immersion and control groups is statistically insignificant. 

Summary of Immersion Effects on TAKS Reading and Mathematics 

Table 6.9 shows the estimated magnitude of the Technology Immersion effect (measured in standard 
deviation units) on TAKS reading and mathematics achievement across student cohorts. Estimated 
effects were generally modest but consistently favored Technology Immersion students compared to 
control. Although students’ cumulative growth in T-score points varied for economically 
disadvantaged and non-disadvantaged students, differences between treatment and control groups 
reported in standard deviation units were identical for both groups (i.e., main effects estimates from 
HLM models). For TAKS reading, the sizes of effects in standard deviation units were very small but 
increased with longer exposure to Technology Immersion (.07 and .02 for Cohorts 1 and 2, 
respectively). Moreover, the positive Technology Immersion effect on students’ reading achievement 
was sustained through ninth grade and approached statistical significance (.08, p < .06). The effects of 
Technology Immersion on TAKS mathematics achievement were larger than for reading. For TAKS 
math, the sizes of effects in standard deviation units for Cohort 2 (.20) and Cohort 3 (.16) were small 
but statistically significant {p < .05). The positive Technology Immersion effect on students’ math 
scores was sustained through ninth grade. The estimated effect for Cohort 1, ninth graders (0.13) was 
similar to the magnitude of the effect in standard deviation units detected at the end of eighth grade. 
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Table 6.9. Model-Based Estimations of Technology Immersion Effects 

on TAKS scores by Subject, Economic Disadvantage Status, and Student Cohort 

| Cumulative Growth 



Immersion 

Control 

Mean 

Standard 


T-score 

T-score 

T- Score 

Deviation 

Assessment/Student Cohort 

Growth 

Growth 

Difference 

Units 

TAKS Reading, Advantaged 





Cohort 1 : 9th graders, post-immersion 

0.76 

-0.06 

0.8 l f 

.08 

Cohort 2: 8th graders, 3 immersion years 

1.10 

0.39 

0.70 

.07 

Cohort 3: 7th graders, 2 immersion years 

0.00 

-0.21 

0.21 

.02 

TAKS Reading, Disadvantaged 





Cohort 1 : 9th graders, post-immersion 

2.22 

1.41 

0.8 1 + 

.08 

Cohort 2: 8th graders, three immersion years 

2.02 

1.32 

0.70 

.07 

Cohort 3: 7th graders, two immersion years 

0.22 

0.01 

0.21 

.02 

TAKS Mathematics, Advantaged 





Cohort 1: 9th graders, post-immersion 

2.14 

0.88 

1.25 

.13 

Cohort 2: 8th graders, 3 immersion years 

1.27 

-0.69 

1.96* 

.20 

Cohort 3: 7th graders, 2 immersion years 

0.74 

-0.83 

1.57* 

.16 

TAKS Mathematics, Disadvantaged 





Cohort 1: 9th graders, post-immersion 

1.63 

0.38 

1.25 

.13 

Cohort 2: 8th graders, three immersion years 

1.81 

-0.15 

1.96* 

.20 

Cohort 3: 7th graders, two immersion years 

0.09 

-1.48 

1.57* 

.16 


Note. Estimated T-score growth for students attending schools with average poverty. Cumulative growth in T- 
score units (mean= 50, standard deviation = 10). Standard deviation units = T-score difference/10. 

V < .10. *p < .05. 


TAKS Social Studies and Science 
Cohort 2 (Technology Immersion) 

Cohort 2 students also completed TAKS science and social studies assessments in spring 2008. For 
science, eighth graders completed a baseline measure as fifth graders in 2005. The TAKS social 
studies assessment is administered for the first time in eighth grade. There was no baseline measure, so 
we used students’ 2005 TAKS reading score as a control for academic achievement. The effects of 
immersion on Cohort 2 students’ science and social studies scores were analyzed using two-level 
FILM models (see Table 6.10). 


Table 6.10. HLM Statistics for Cohort 2 (Eighth Graders): 

Effect of Immersion (Fixed) on TAKS Science and Social Studies Achievement 



TAKS Science 

TAKS Social Studies 


N = 3,268 

N = 3,268 


Gamma 


Gamma 



Coefficient 

t-value 

Coefficient 

t-value 

Intercept (TAKS T score) 

50.839 

81.14*** 

51.850 

53.95*** 

Immersion 

0.475 

0.65 

0.006 

0.01 

School poverty 

0.033 

1.41 

0.030 

1.03 

Female 

-0.036 

-0.13 

-2.049 

-6.75*** 

African American 

-1.902 

-2.43* 

-1.938 

-2.63** 

Hispanic 

-1.484 

-3.01** 

-1.058 

-2.04* 

Economic disadvantage 

-1.611 

-4.39*** 

-1.616 

_3 79 *** 

2005/2007 TAKS T score a 

0.641 

28.00*** 

0.537 

27.88*** 


a The pre-measure for science is the 2005 TAKS science score; the pre-measure for TAKS social studies 
is the 2005 TAKS reading score. *p < .05. **p < .01. ***p < .001. 
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In the student-level model (Level 1), students’ 2008 T scores were regressed on students’ baseline 
scores, gender, minority status, and economic status. A school-level model (Level 2) was used to 
determine whether students in immersion schools had higher TAKS science and social studies scores 
than control-group students in spring 2008, after adjusting for initial achievement, student 
demographic characteristics, and school poverty. The immersion variable identified the comparison 
groups (a value of 1 for an immersion school and 0 for control). School poverty was a continuous 
variable, with a mean of 68.5%, indicating the percentage of economically disadvantaged students in a 
school. 

Analyses for TAKS science and social studies involved 1,571 immersion students and 1,697 control 
students, with similar proportions of students included in analyses across groups (60.9% for 
immersion and 59.4% for control). Science outcomes show that after controlling for Cohort 2 students’ 
prior TAKS achievement, demographic characteristics, and the level of school poverty, there was no 
statistically significant effect of immersion on eighth graders’ 2008 TAKS science T scores. The 
immersion effect was positive (0.48 T-score point) but not by a significant margin. Results for TAKS 
social studies, showed that after controlling for students’ 2007 TAKS reading achievement, 
demographic characteristics, and the level of school poverty, there was no statistically significant 
difference between the 2008 TAKS social studies T scores for immersion and control students. In 
contrast to science, the immersion effect on social studies was essentially zero (0.01 T-score point). 
Across both immersion and control schools, economically disadvantaged students had significantly 
lower TAKS science scores (-1.61 T-score points) and social studies scores (-1.62 T-score points) than 
their more affluent counterparts. And, minority students (African American and Hispanic) had 
significantly lower scores than other students. Unexpectedly, female students had significantly lower 
social studies scores than males (-2.05 T-score point). 

TAKS Writing 

Cohort 3 (Technology Immersion) 

Cohort 3 students completed the TAKS writing assessment as fourth graders in 2005 and again as 
seventh graders in 2008. We used a two-level HLM model to estimate the effects of immersion on 
students’ writing scores (see Table 6.11). 

Table 6.11. HLM Statistics for Cohort 3 (Seventh Graders): 

Effects of Immersion (Fixed) on TAKS Writing Achievem ent 


Dependent variable 
and predictor 

TAKS Writing 
N = 3,088 

Gamma 

Coefficient 

f-value 

Intercept (TAKS T score) 

51.390 

102.75*** 

Immersion 

-0.723 

-1.25 

School poverty 

0.003 

0.17 

Female 

-0.218 

-0.78 

African American 

-1.099 

-2.60* 

Hispanic 

-0.407 

-0.75 

Economic disadvantage 

-2.001 

-5 91*** 

Spring 2005 T score 

0.644 

25.61*** 


*p < .05. **p < .01. ***p < .001. 


In the student-level model (Level 1), students’ 2008 writing T scores were regressed on 2005 writing 
scores (data from two years prior to students’ involvement in the immersion project), gender, minority 
status, and economic status. A school-level model (Level 2) predicted whether students in immersion 


70 




schools had higher 2008 TAKS writing T scores than control-group students, after adjusting for initial 
achievement, student demographic characteristics, and school poverty. HLM analyses involved 1,703 
immersion students and 1,969 control students. Results show that after controlling for Cohort 3 
students’ pretest writing scores, student demographic characteristics (gender, ethnicity, economic 
status), and school poverty level, there was no statistically significant difference in the 2008 TAKS 
writing T scores for students in immersion and control schools. The immersion effect on writing was 
negative (about -0.72 T-score point lower than for control-group students). Across both immersion and 
control schools, the demographic characteristics of students were strongly associated with TAKS 
writing achievement. African American students (-1.10 T-score points) had significantly lower writing 
scores than other ethnic groups, and economically disadvantaged students had significantly lower 
scores (-2.00 T-score points) than their more affluent peers. 

Conclusions 

In the fourth and final project year, we examined the effects of Technology Immersion on Cohort 2 
students (eighth graders who attended middle schools for three years). Cohort 3 students (seventh 
graders who attended middle schools for two years), and Cohort 1 students (ninth graders who 
attended middle schools for three years and then enrolled in mainly traditional high schools). Key 
findings are the following. 

• TAKS reading. After controlling for student and school poverty, there were no statistically 
significant effects of immersion on the TAKS reading growth rates for either Cohort 2 
students or Cohort 3 students. The immersion effects were positive but very small. Across 
both student cohorts, positive mean growth trajectories showed that economically 
disadvantaged students and students in schools with above average levels of poverty grew in 
reading achievement at faster rates than their more affluent peers. For Cohort 1, post- 
immersion and control ninth graders attending high schools, there was a positive enduring 
effect of Technology Immersion on treatment students’ TAKS reading growth rate that 
approached statistical significance (p < .0.06). 

• TAKS mathematics. After controlling for student and school poverty, Technology Immersion 
had a statistically significant effect on the TAKS mathematics growth rates for both Cohort 2 
and Cohort 3 students. The TAKS mathematics scores of immersion students increased across 
years, whereas scores for control students decreased. For Cohort 1 , post- immersion and 
control ninth graders attending high schools, there was a positive but statistically 
nonsignificant sustaining effect of Technology Immersion on TAKS mathematics 
achievement. 

• TAKS science. After controlling for prior science achievement, demographic characteristics, 
and school poverty, there was no statistically significant effect of immersion on Cohort 2, 
eighth graders’ 2008 TAKS science scores. The estimated immersion effect was positive but 
very small. 

• TAKS social studies. After controlling for Cohort 2, eighth graders’ reading achievement 
(seventh grade), demographic characteristics, and school poverty, there was no statistically 
significant effect of immersion on 2008 TAKS social studies scores. The estimated immersion 
effect was virtually zero (.006 T-score point). 

• TAKS writing. After controlling for Cohort 3 seventh graders’ pretest writing scores (fourth 
grade), demographic characteristics, and campus poverty, there was no statistically significant 
difference in the TAKS writing scores for immersion and control students. The estimated 
immersion effect was negative but very small. 
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7. Association between Implementation and Academic Outcomes 


Chapter 3 provided findings on the implementation of Technology Immersion for the second through 
fourth project years (2005-06, 2006-07, and 2007-08, respectively). Implementation was measured as 
the fidelity with which Technology Immersion components and related elements attained the model’s 
envisioned ideal (see implementation indicators in Exhibit 7.1). Mean immersion standard scores 
revealed small yearly increases across most of the implementation support components (Leadership, 
Teacher Support, Parent and Community Support, and Professional Development) as well as increases 
in teachers’ overall level of Classroom Immersion. Conversely, the level of Student Access and Use 
declined across years, with notable fourth-year decreases in the frequency of Core-Content Learning 
and the extent of laptop use for Home Learning. 


Exhibit 7.1. Implementation Indicators for Technology Immersion 

Immersion Support Index is an aggregate score for school-level indicators of support for Technology 
Immersion. 

■ Leadership is a measure of administrative leadership for technology. 

■ Teacher Support is a measure of teachers’ commitment to immersion. 

■ Parent and Community Support is a measure of support for the school’s technology efforts. 

■ Technical Support is a measure of the extent to which technical support alleviates problems 

that create barriers to immersion. 

■ Professional Development is an aggregate indicator of the quality of campus professional 

development as measured by four elements: Contact Hours , Classroom Support , Content 
Focus, and Coherence. 

Classroom Immersion Index is an aggregate score for teacher-level immersion indicators. 

■ Technology Integration is a measure of a teacher’s ideological orientation towards classroom 

Technology Immersion. 

* Learner-Centered Instruction is a measure of a teacher’s ideological orientation towards 
student-centered learning practices. 

■ Student Activities is a measure of the frequency of students’ use of technology resources in 

a teacher’s classroom. 

■ Communication is a measure of a teacher’s technology-based communications with students, 

parents, and peers. 

■ Professional Productivity is a measure of a teacher’s use of technology for professional 

activities. 

Student Access and Use Index is an aggregate score for student-level immersion indicators. 

■ Laptop Access Days is a measure of the extent to which a student has access to a laptop 

throughout the school year. 

■ Core-Content Learning is a measure of the frequency that a student reports using technology 

for learning in core-subject classes. 

■ Home Learning is a measure of the extent that a student uses a laptop for core-subject 

homework (language arts [reading/writing], social studies, science, and math) or to play 
games to learn outside of school. 

Implementation Index is an implementation score for each school, which is an aggregate score for 
the three implementation components described above. 

Note. Implementation indices are z scores with a mean of 0 and standard deviation of 1 .0. 


Implementation evidence for the fourth year, similar to previous years, revealed wide variation across 
schools and classrooms. For the fourth year, we estimated that about a quarter of middle schools (6) 
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had a much stronger presence of the immersion components that more nearly approximated full 
implementation standards. Given variations in implementation from school-to-school and from 
classroom-to-classroom, we report in this chapter on the relationships between implementation levels 
and student academic achievement. For analyses, we used standardized implementation indicators 
(z scores with a mean of 0 and a standard deviation of 1.0) that could be analyzed individually or 
aggregated to generate component scores and an overall implementation score. Analyses involved 
indicators that assessed school supports for immersion (Immersion Support Index), the extent of 
teachers’ classroom immersion (Classroom Immersion Index), and the extent of students’ technology 
access and use (Student Access and Use Index). 

Analysis Method 

We used a series of two-level hierarchical linear models (HLM), in which students were nested within 
teachers’ classrooms, to investigate whether the levels of implementation for two teacher-related 
implementation components (Immersion Support Index, Classroom Immersion Index) and one 
student-specific component (Student Access and Use Index) were significant predictors of students’ 
TAKS reading and mathematics scores. We analyzed the effects of implementation on academic 
achievement for Cohorts 2 and 3 students. 

In the student-level model (Level 1), 2008 TAKS T scores were regressed on 2007 TAKS T scores, the 
Student Access and Use Index (z score), economic status (0 if not disadvantaged, 1 if disadvantaged), 
African American status (0 if not African American, 1 if African American), Hispanic status (0 if not 
Hispanic, 1 if Hispanic) and gender (0 if male, 1 if female). The teacher-level model (level 2) 
investigated whether the Immersion Support Index (average campus z score) and Classroom 
Immersion Index (individual teacher z score) predicted higher 2008 TAKS scores, after adjusting for 
school poverty, students’ prior achievement and demographic characteristics, and Student Access and 
Use. We also investigated whether Student Access and Use predicted higher 2008 TAKS scores, after 
adjusting for initial achievement, student demographic characteristics, school poverty. Immersion 
Support, and Classroom Immersion. School poverty was a continuous variable indicating the 
percentage of economically disadvantaged students in a school, with a mean of 71.0%. Analyses for 
Cohorts 2 and 3, respectively, involved approximately 1,100 students who were enrolled continuously 
in schools during three project years and 1,200 students who were continuously enrolled for two 
project years. 

TAKS Reading 

Estimates of the effects of implementation on Cohorts 2 and 3 students’ 2008 TAKS reading T scores 
are presented in Table 7.1. At the teacher level, we investigated whether the strength of reading 
teachers’ campus support for implementation (Immersion Support) and their reported levels of 
Classroom Immersion were predictors of students’ reading achievement. None of the teacher-level 
implementation measures were statistically significant predictors of TAKS reading scores. After 
controlling for student variables (prior achievement, demographic characteristics, Student Access and 
Use) and other teacher variables (school poverty and Classroom Immersion), Immersion Support was 
a positive predictor of Cohort 2 eighth graders’ reading achievement but a negative predictor of 
Cohort 3 students’ reading scores. Reading teachers’ level of Classroom Immersion was a consistently 
positive predictor of students’ TAKS reading achievement. After adjusting for other variables in the 
analysis, Cohorts 1 and 2 students who had reading teachers with average levels of Classroom 
Immersion had slightly higher TAKS reading T-scores (0.22 and 0.21 points, respectively) than 
students with teachers having below average Classroom Immersion scores. 
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Table 7.1. Hierarchical Regression Models Predicting the Effects of Implementation Components on 
TAKS Reading Achievement 



Cohort 2 
Eighth Graders 
N= 1,101 

Cohort 3 
Seventh Graders 
N= 1,168 

Predictor 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

?-value 

Intercept 

50.264 

128.56*** 

49.204 

99.61*** 

Teacher-level predictors 

School poverty 

-0.016 

-1.18 

-0.021 

-1.15 

Immersion Support 

0.064 

0.52 

-0.315 

-1.16 

Classroom Immersion 

0.215 

1.45 

0.211 

0.73 

Student-level predictors 

Spring 2007 T score 

0.689 

21.38*** 

0.666 

34.78*** 

Student Access and Use 

0.466 

1.38 

0.791 

1.93 1 

Female 

0.674 

1 .95 1 

0.216 

0.52 

African American 

-1.862 

-2.12* 

-1.649 

-2.18* 

Hispanic 

-0.164 

-0.43 

-0.518 

-0.83 

Eco. Disadvantaged 

-0.601 

-1.33 

-0.875 

-1.63 


< .10. *p < .05. **p < .01. ***p < .001. 

Note. Numbers of reading teachers: Cohort 2 = 37 and Cohort 3 = 34. 


In contrast to teacher-level predictors, the level of Student Access and Use (of technology) was a 
stronger predictor of reading achievement, but the effect was non-significant for Cohort 2 and just 
marginally statistically significant for Cohort 3. Hence, after controlling for students’ prior reading 
achievement, demographic characteristics, and teacher-level variables (school poverty and 
implementation components), the sizes of the Student Access and Use effect on TAKS reading 
achievement for Cohorts 2 and 3 students were 0.47 and 0.79 T- score points, respectively. 

Additionally, we conceptualized Student Access and Use as having multiple elements (Laptop Access 
Days, Core-Content Learning, and Home Learning), and thus, were interested in separately predicting 
variation for each element. Table 7.2 provides statistics for the HLM models used to predict each of 
the three elements. Findings revealed that Home Learning — which measured the extent of a student’s 
laptop use outside of school for homework in each of the four core-subject areas and for learning 
games — was the strongest implementation predictor of reading achievement. The Home Learning 
effect on TAKS reading scores was positive for Cohort 2 (0.30 T-score point) and statistically 
significant and positive for Cohort 3 (0.99 T-score point). As an example, after controlling for all of 
the other variables in the analysis, an economically advantaged, non-minority, male seventh grader 
with a score one standard deviation above average for Home Learning (z - 1.00), had a 0.99 T-score 
point higher TAKS reading score. Moreover, with each additional standard deviation increase in 
Home Learning, students’ reading achievement increased even more. 

In contrast to Home Learning, the number of days during the school year that students had laptops 
available for use (Laptop Access Days) was a mixed and non- significant predictor of students’ reading 
achievement. The frequency that students reported using their laptops in their four core-subject classes 
(Core-Content Learning) was a non-significant and negative predictor of achievement for both Cohorts 
2 and 3, after controlling for other variables in the analysis. 
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Table 7.2. Hierarchical Regression Models Predicting the Effects of Implementation Components 
(including Elements of Student Access and Use) on TAKS Reading Achievement 



Cohort 2 
Eighth Graders 

Cohort 3 
Seventh Graders 

Predictor 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

50.326 

126.61*** 

49.274 

100.01*** 

Teacher-level predictors 

School poverty 

-0.019 

-1.32 

-0.032 

-1.88 1 

Immersion Support 

0.105 

0.85 

-0.263 

-0.98 

Classroom Immersion 

0.239 

1.55 

0.365 

1.41 

Student-level predictors 

Spring 2007 T score 

0.686 

20.83*** 

0.658 

34.87*** 

Laptop Access Days 

0.195 

0.89 

-0.186 

-0.73 

Core-Content Learning 

-0.098 

-0.41 

-0.344 

-1.54 

Home Learning 

0.304 

1.27 

0.985 


Female 

0.641 

1.88 1 

0.152 

0.37 

African American 

-1.872 

-2.12* 

-1.624 

-2.27* 

Hispanic 

-0.202 

-0.52 

-0.648 

-1.07 

Eco. Disadvantaged 

-0.627 

-1.39 

-0.765 

-1.41 


< .10. *p < .05. **p < .01. ***p < .001. 

Note. Numbers of reading teachers: Cohort 2 = 37 and Cohort 3 = 34. 


TAKS Mathematics 

We also estimated the effects of implementation on students’ 2008 TAKS mathematics T scores. Like 
reading, we examined implementation effects for students and teachers (Table 7.3). 


Table 7.3. Hierarchical Regression Models Predicting the Effects of Implementation Components 
and TAKS Mathematics Achievement 



Cohort 2 
Eighth Graders 
A =999 

Cohort 3 
Seventh Graders 
N= 1.165 

Predictor 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

50.542 

108.15*** 

50.313 

101.20*** 

Teacher-level predictors 

School poverty 

0.011 

0.57 

-0.010 

-0.40 

Immersion Support 

-0.168 

-0.55 

0.026 

0.07 

Classroom Immersion 

0.451 

1.03 

-0.614 

-1.49 

Student-level predictors 

Spring 2007 T score 

0.702 

32.65*** 

0.730 

35.24*** 

Student Access and Use 

0.303 

1.01 

0.505 

1.39 

Female 

-0.338 

-0.86 

0.509 

1 .78* 

African American 

-1.528 

-2.19* 

-2.475 

-2.74** 

Hispanic 

-0.436 

-1.08 

-0.998 

-1.73 1 

Eco. Disadvantaged 

-0.607 

-1.41 

-0.632 

-1.26 


1 p < .10. *p < .05. **p < .01. ***p < .001. 

Note. Numbers of mathematics teachers: Cohort 2 = 37 and Cohort 3 = 38. 


Comparable to reading, none of the teacher-level implementation indicators was a statistically 
significant predictor of students’ TAKS mathematics scores. After controlling for other variables in 
the analysis. Immersion Support was a negative predictor of Cohort 2 students’ mathematics 
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achievement and a positive predictor of achievement for Cohort 3. After statistical adjustments for the 
other variables in the analysis, mathematics teachers’ reported Classroom Immersion level was a 
positive predictor of TAKS math achievement for Cohort 2 but a negative predictor for Cohort 3. In 
contrast to teacher-related implementation indicators, students’ reported level of Student Access and 
Use was a consistently positive predictor of 2008 TAKS mathematics T scores for each of the student 
cohorts, although not by a statistically significant margin. Controlling for students’ prior math 
achievement, demographic characteristics, and teacher-level variables (implementation components as 
well as school poverty), the sizes of the Student Access and Use effects were 0.30 and 0.5 1 T-score 
points for Cohorts 2 and 3 students, respectively. 

To gain a greater understanding of the association between students’ reported technology access and 
use and mathematics achievement, we used HLM to predict math achievement for each of the three 
Student Access and Use elements (Laptop Access Days, Core-Content Learning, and Home Learning). 
Results in Table 7.4, similar to TAKS reading outcomes, show that the extent to which students 
reported using their laptops for Home Learning was a statistically significant predictor of TAKS 
mathematics scores. The Home Learning effect on mathematics achievement was slightly stronger for 
Cohort 3 (0.48 T-score point) compared to Cohort 2 (0.32 J-score point). As an example, after 
controlling for the other variables, an economically advantaged, non-minority, male seventh grader 
with a Home Learning score about one standard deviation above average (z = 0.99), had a 0.48 T-score 
point higher TAKS mathematics score. As the extent of laptop use for Home Learning increased, 
mathematics achievement increased incrementally. 


Table 7.4. Hierarchical Regression Models Predicting the Effects of Implementation Components 
(Including Elements of Student Access and Use) on TAKS Mathematics Achievement 



Cohort 2 
Eighth Graders 

Cohort 3 
Seventh Graders 

Predictor 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

50.588 

109.51*** 

50.343 

98.36*** 

Teacher-level predictors 

School poverty 

0.007 

0.35 

-0.013 

-0.51 

Immersion Support 

-0.159 

-0.52 

0.214 

0.57 

Classroom Immersion 

0.416 

0.98 

-0.584 

-1.42 

Student-level predictors 

Spring 2007 T score 

0.698 

33.54*** 

0.725 

35.57*** 

Laptop Access Days 

0.019 

0.12 

0.181 

0.72 

Core-Content Learning 

-0.146 

-0.72 

-0.322 

-1.70* 

Home Learning 

0.324 

1.74* 

0.482 

2.07* 

Female 

-0.376 

-0.98 

0.438 

1.48 

African American 

-1.542 

-2.23* 

-2.418 

-2.70** 

Hispanic 

-0.442 

-1.10 

-1.017 

-1.78* 

Eco. Disadvantaged 

-0.627 

-1.48 

-0.605 

-1.19 


T p < .10. *p < .05. **p < .01. ***p < .001. 

Note. Numbers of mathematics teachers: Cohort 2 = 37 and Cohort 3 = 38. 


In contrast to Home Learning, students’ reported number of Laptop Access Days was a positive but 
non-significant predictor of TAKS mathematics achievement across the two student cohorts. 
Conversely, the frequency of laptop use for Core-Content Learning in classrooms was negatively 
associated with students’ mathematics achievement for both cohorts, with the negative relationship 
marginally statistically significant for Cohort 3. 
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Conclusions 


In this chapter we described associations between the implementation of Technology Immersion 
components and students’ academic achievement. Key findings are the following. 

• Data analyses for individual students and their teachers showed that the campus measure of 
Immersion Support and reading and mathematics teachers’ reported levels of Classroom 
Immersion were inconsistent predictors of students’ TAKS reading and mathematics 
achievement. 

• Conversely, the level of Student Access and Use (of technology) was a consistently positive 
although not statistically significant predictor of students’ TAKS reading and mathematics 
achievement for Cohort 2 (eighth graders) and Cohort 3 (seventh graders). Student Access and 
Use was a marginally statistically significant predictor of TAKS reading scores for Cohort 3 
students. 

• Of the three elements of Student Access and Use, students’ use of their laptops for Home 
Learning — a measure of the extent to which a student used a laptop outside of school for 
homework in the four core-subject areas and for learning games — was the strongest 
implementation predictor of TAKS reading and mathematics achievement across both cohorts. 

• For Cohort 2 (eighth graders), the extent of Home Learning was a positive but non-statistically 
significant predictor of students’ TAKS reading achievement and a positive and marginally 
significant predictor of TAKS mathematics achievement. For Cohort 3 (seventh graders), the 
extent of Home Learning was a positive and statistically significant predictor of both TAKS 
reading and mathematics achievement. 
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8. Conclusions and Implications 


The fourth-year evaluation provides final results on the effects of the Technology Immersion model 
(i.e., a laptop computer for every student and teacher, wireless access throughout the campus, 
curricular and assessment resources, professional development, and ongoing technical and pedagogical 
support) on schools, teachers, and students. This report combines information gathered during the 
fourth project year (2007-08) with data from the first through third implementation years (2004-05 
through 2006-07). 

The study’s quasi-experimental research design has allowed inferences about causal effects through 
comparisons between 21 treatment schools and 21 control schools. Schools included Grades 6 to 8 
middle schools drawn from rural, suburban, and urban locations across Texas. Middle schools were 
typically small (about 400 students, on average); however, enrollments varied widely (from 83 to 
1,447 students). About two-thirds of schools were located in small or very small Texas districts (less 
than 3,000 students), and about a third were in very large districts (10,000 or more students). Students 
in the study were mostly economically disadvantaged (67%) and they were racially and ethnically 
diverse (roughly 58% Hispanic, 7% African American, and 36% White). 

The study focused on three student cohorts in the fourth year. Cohort 2 included eighth graders (2,578 
treatment and 2,858 control students) who finished their third immersion year; Cohort 3 included 
seventh graders (2,547 treatment and 2,845 control students) who concluded their second year. In the 
fourth year, we also examined achievement data for Cohort 1 students who had attended Technology 
Immersion and control schools from sixth-through-eighth grade and then enrolled in traditional high 
schools, which typically did not provide individual laptops for students (2,469 treatment students and 
2,748 control-group students). 

Study Limitations 

The sample selection process and matching procedures used with the quasi-experimental design 
produced a sample of schools with good internal validity, in that there were no large, statistically 
significant treatment-control group differences. Baseline data confirmed that the comparison groups 
were reasonably well matched, but we have also used statistical controls to adjust for differences that 
could have arisen from sampling variability. A threat to internal validity was introduced in the third 
year when control schools began to plan for Technology Immersion. Most of the control teachers 
received laptops, instructional resources, and more intensive professional development in the third 
year, and in the fourth project year, some students at control schools received laptops. In particular, 
records submitted by schools indicated that about 260 eighth graders (9%) and 480 seventh graders 
(17%) at control schools received individual laptops during the fourth year. The introduction of 
Technology Immersion components in control schools may bias fourth-year results. Given the positive 
associations between Technology Immersion and teacher and student outcomes identified across three 
project years, reported findings may underestimate the magnitude of the treatment effect. On the other 
hand, given that some studies show schools’ academic outcomes may decline during initial 
implementation phases (Borman, Hewes, Overman, & Brown, 2003; Borman, 2005), the introduction 
of Technology Immersion components in control schools might positively bias outcomes for treatment 
schools. 
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Generalization of findings to a broader population (external validity) is a primary study limitation. 
Compared to Texas middle-school students as a whole, students in the sample schools are substantially 
more Hispanic and less White and African American. Middle schools are also smaller than the 
statewide average (402 students versus 667). Schools also are located either in small or very small 
districts (64%) or large districts (36%), which is different from the statewide distribution of schools. 
Additionally, for many variables, the study relies on self-reported data from surveys of teachers and 
students — thus, some findings on changes in proficiencies and practices reflect respondents’ 
perceptions. Nonetheless, the triangulation of evidence from multiple sources (surveys, classroom 
observations, state demographic and test databases, multiple student cohorts) verifies the robustness of 
findings. Despite cited limitations, researchers are confident that reported effects can be attributed to 
the treatment. 

Major Fourth-Year Findings 

Like previous years, outcomes represent the effects of Technology Immersion for schools that 
generally had less than full implementation levels. Although the overall quality of schools’ 
implementation improved slightly in the fourth year, we estimated that just a quarter of middle schools 
(6) achieved substantial immersion levels, whereas the remaining schools (15) had minimal to partial 
immersion levels. Major findings from the fourth year are described in the following sections. A final 
section discusses the quality of Technology Immersion implementation, prospects for sustainability of 
the model, and implications for educational policy. 

Effects of Technology Immersion on Teachers and Teaching 

We assessed the effects of Technology Immersion on teachers and teaching by examining teachers’ 
rates of growth on mediating variables across five time points (fall 2004 through spring 2008). 
Analyses involved 2,137 teachers, including 1,046 in immersion schools and 1,091 in control schools. 
Even though control teachers benefited in the third and fourth years from initial steps toward 
implementation of the Technology Immersion model, we found that being part of a Technology 
Immersion school across four years affected teachers positively in a number of ways. 

Teachers in Technology Immersion schools grew in technology proficiency, their use of 
technology for professional productivity, and their use of technology for student learning 
activities at significantly faster rates than control teachers. Although both treatment and control 
teachers became more technology literate over time, immersion teachers grew at a significantly faster 
pace. Teachers’ self-assessments of Technology Proficiency across time indicated that immersion 
teachers were increasingly more accomplished than control teachers in areas involving technology 
operations (e.g., using software applications) and pedagogical skills (e.g., creating lessons plans 
integrating technology). Estimated yearly growth trajectories for immersion teachers in schools with 
average student poverty, compared to control, were nearly twice as steep (0.29 and 0.15 scale-score 
points per year, respectively, on a 7-point scale). Consistent with previous years, teachers who taught 
at schools with higher levels of student poverty grew in technology proficiency at significantly slower 
rates than their peers in more advantaged schools. As the level of school poverty increased, the 
proficiency gap between teachers in higher and lower poverty schools widened. 

Teachers at Technology Immersion schools also grew in their use of technology to enhance their 
Professional Productivity at a significantly faster rate than control teachers (0. 1 8 and 0.11 scale-score 
points per year, respectively, on a 5-point scale). However, the gap between teacher groups narrowed 
substantially in the third and fourth years as teachers in control schools acquired more technology 
resources. Consequently, teachers at both treatment and control schools were using technology more 
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frequently for puiposes such as making electronic presentations, administering online assessments, 
and accessing model lesson plans. 

Teachers at Technology Immersion schools, who continued to have greater classroom access to 
computers than control teachers, increased the frequency of their students’ Classroom Activities 
involving technology at a more rapid pace (0.17 scale-score point per year on a 5-point scale versus 
0.07 point, for teachers in schools with average poverty). Although student activities with technology 
increased steadily across years in immersion classrooms, fourth-year averages showed that students as 
a whole still used various technology resources infrequently (i.e., about once or twice a month, 

M = 2.72). Mean statistics, however, obscured the substantial teacher-to-teacher variation in the 
frequency of students’ technology activities both across and within subject areas. Similar to previous 
years, English language aits, science, and social studies teachers had students use technology 
considerably more often than mathematics teachers. 

Teachers at Technology Immersion schools expressed significantly stronger ideological 
associations across years with technology integration and learner-centered practices. Teachers at 
both immersion and control schools became more positive towards innovative technology practices 
across years, but immersion teachers altered their beliefs at a significantly faster rate. For Technology 
Integration, the mean estimated growth for immersion teachers in schools with average poverty was 
0.40 scale-score point per year compared to 0.30 for control teachers (on a 7-point scale). Thus, 
immersion teachers increasingly employed actions supporting curricular and instructional infusion of 
technology, such as promoting students’ authentic problem solving or critical thinking through 
technology. Immersion teachers also expressed increasingly stronger affiliations with constructivist or 
learner-centered practices, such as having students establish individual learning goals and emphasizing 
experiential learning. The estimated yearly growth rates in learner-centered practices for immersion 
and control teachers in average poverty schools were 0.31 and 0.20 scale-score points, respectively, on 
a 7-point scale. 

The introduction of Technology Immersion components in schools affected teachers’ perceptions 
of the school’s culture as well as the frequency of teachers’ collegial interactions. Our study, 
similar to others, shows that the introduction of technology resources can be a catalyst for school 
change (e.g.. Baker et al., 1994; Dwyer, 1994). Across the first two project years, teachers in 
immersion schools compared to control reported significantly stronger administrative leadership for 
technology, greater parent and community support for technology, a more innovative school culture, 
and increased collaborative interactions with colleagues that supported improvements in instructional 
practices (such as coaching and mentoring, developing lesson plans collectively, exchanging 
information about students). With the introduction of teacher laptops and other technology resources 
in control schools during the third and fourth project years, differences between the views of treatment 
and control teachers dissipated. Similar to treatment teachers, the teachers in control schools began to 
perceive stronger administrative leadership and collective teacher support for technology. Thus, as 
control teachers experienced components of Technology Immersion, they also began to view their 
schools’ technology environments as more innovative and supportive. Still, teachers at Technology 
Immersion schools who had access to professional development and technology resources over a 
longer period of time continued in the fourth year to report more frequent collaborative interactions 
with their colleagues that supported instructional practices involving technology (e.g., developing 
lesson plans or exchanging information about students) compared to control teachers. 

Evidence from classroom observations suggested that laptop computers and digital resources 
allowed students in Technology Immersion schools to experience somewhat more intellectually 
demanding work. The Technology Immersion model assumes that technology resources will promote 
students’ higher level thinking through more challenging and relevant learning activities that support 
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academic achievement. Accordingly, across four years, researchers have observed lessons in 
immersion and control teachers’ classrooms and rated the Intellectual Challenge of lessons (Newmann 
et al., 1995). Observations of core-subject classes (English language arts/reading, mathematics, 
science, and social studies) revealed no statistically significant differences between the overall 
Intellectual Challenge of immersion and control teachers’ instruction. However, effect sizes measuring 
differences between treatment and control teachers’ instruction generally showed positive effects 
favoring treatment teachers, especially for the domains measuring Higher Order Thinking (e.g., 
synthesizing, generalizing, explaining) and Depth of Knowledge (e.g., thorough exploration of a topic 
that produces complex understandings). Longitudinal trends indicated that the introduction of 
technology resources had a similarly positive influence on control teachers’ instruction. Still, despite 
positive progress, results for all observed classrooms indicated that lessons in middle school core- 
subject classes generally failed to intellectually challenge students, with average ratings in the fourth 
year of about 2 on the 5-point challenge scale. 

Effects of Technology Immersion on Students and Learning 

In the fourth project year, we measured student mediating variables across four time periods for 
Cohort 2 eighth graders (fall 2005 and spring 2006, 2007, and 2008) and three periods for Cohort 3 
seventh graders (fall 2006 and spring 2007 and 2008). Analyses for Cohorts 2 and 3, respectively, 
included 2,167 immersion and 2,361 control students, and 2,073 immersion and 2.372 control 
students. Controlling for important school and student characteristics, key findings included the 
following. 

Economically advantaged and disadvantaged students in Technology Immersion schools became 
significantly more technology proficient than their counterparts in control schools. Economically 
disadvantaged students in immersion schools reached proficiency levels that matched the skills 
of advantaged control students. Across implementation years and student cohorts, students in 
Technology Immersion schools have made significantly greater progress in mastering the Texas 
Technology Applications standards than control students (e.g., sending an email attachment, creating a 
presentation, managing documents, using spreadsheets, and keeping track of websites). For Cohort 2 
(eighth graders), both economically advantaged and disadvantaged immersion students grew in 
proficiency at faster rates (0.35 and 0.35 scale-score points per year, respectively, on a 5-point scale) 
than their control-group counterparts (0.26 and 0.26 scale-score points). For Cohort 3 (seventh 
graders) the immersion effect was even stronger, with both economically advantaged and 
disadvantaged immersion students growing significantly faster in proficiency (0.43 and 0.42 scale- 
score points per year, respectively) than control-group students (0.27 and 0.26 scale-score points, 
respectively). As a consequence, economically disadvantaged students in Technology Immersion 
schools reached levels of technical proficiency that equaled the proficiencies of advantaged students in 
control schools. 

Students in Technology Immersion schools used technology applications more often in their 
core-subject classes and they interacted more often with their peers in small-group activities. 

Similar to previous evaluation years, students in immersion schools used technology applications 
significantly more often in their core-subject classes than control students. For Cohorts 2 and 3, the 
yearly growth rates in Classroom Activities for economically advantaged and disadvantaged 
immersion students ranged from 0.19 to 0.43 scale-score points (on a 5-point scale), compared to 0.10 
to 0.21 points for comparable control-group students. Despite significant yearly increases, fourth-year 
scores (similar to teachers’ reports) indicated that students, on average, used various technology 
resources infrequently in core classes (about once or twice a month). 

Along with greater uses of classroom technology, students in immersion schools also had more 
frequent opportunities to learn in small groups with their classmates. Seventh and eighth graders in 
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immersion schools had increasing opportunities for small-group work with their peers, whereas their 
counterparts in control schools reported less frequent small-group activities as they advanced to higher 
grade levels. Thus, as immersion teachers acquired new resources, many teachers began to alter their 
instructional practices and started to organize student classroom activities differently. 

As laptops aged over four years, students at Technology Immersion schools, compared to 
control, reported more technical problems when they used computers at school. In the fourth 
year, students in Technology Immersion schools reported technical problems with computers at more 
than twice the rates reported by control students. Eighth graders (Cohort 2) who often inherited 
second-hand laptops and had used those laptops across three school years, and seventh graders 
(Cohort 3) who also often inherited and used worn laptops, reported significantly more technical 
problems than control-group students. Mean scores reported by students in spring 2008 indicated that 
various technical problems occurred rarely (a few times a year) or just sometimes (once or twice a 
month). However, increased problems with deteriorating laptops substantially increased the workloads 
of technical-support staff, which in many cases were already overburdened with technical demands. 

Across four evaluation years, there was no evidence linking Technology Immersion with student 
self-directed learning or their general satisfaction with schoolwork. Some research studies have 
suggested that the independent and self-guided learning afforded through one-to-one technology will 
positively affect students’ personal self-direction (e.g., Garrison, 1997; Raghavan, Sartoris, & Glaser, 
1997; Zimmerman, 1989). However, findings from three student cohorts across four evaluation years 
showed there was no statistically significant effect of Technology Immersion on student Self-Directed 
Learning, as measured by the Style of Learning Inventory. Across years, as both immersion and 
control students progressed from lower to higher grade levels, their responses to statements measuring 
self-direction (e.g., goal setting, self-efficacy beliefs, and intrinsic effort) revealed significantly 
negative growth trends. Thus, students reported less self -regulated learning behaviors across time. We 
also measured students’ levels of satisfaction with the kinds of work they did in classes (e.g., 
meaningfulness of class work) and with their perceived relevance of schoolwork (e.g., usefulness for 
the future). We found that the excitement of having laptops appeared to elevate the satisfaction of sixth 
graders during their first middle-school year. However, as students advanced to seventh and eighth 
grade, there was no significant difference in the levels of satisfaction with schoolwork expressed by 
treatment and control students. Across all middle schools, students’ responses to statements related to 
their understanding about why they do certain things in classes, the extent that meaningful work makes 
them try harder, and beliefs that class work will help them as adults or in future jobs moved toward 
uncertainty or disagreement and away from agreement. 

Across four years, students in Technology Immersion schools consistently had fewer disciplinary 
actions than control-group students. Consistent with previous research linking one-to-one 
computing with reduced student discipline problems (e.g., Baldwin, 1999; Barron, Hogarty, Kromery, 
& Lenkway, 1999; MEPRI, 2003; Stevenson, 1998), results replicated across three student cohorts for 
this study showed that students at Technology Immersion schools had significantly fewer disciplinary 
problems. Disciplinary Action Reports submitted to the Texas Education Agency (TEA) for each 
student during the 2007-08 school year, similar to the previous three years, showed that immersion 
students had proportionately fewer disciplinary actions than their counterparts in control schools. In 
the fourth year, Cohorts 2 and 3 immersion students had an average of 0.54 and 0.45 disciplinary 
actions per student, respectively, compared to 0.76 and 0.71 per-student averages for control students. 
Even though the effect sizes for the mean differences between groups were small (-. 1 1 and -. 13), the 
reduction in disciplinary actions in middle schools may have practically important benefits due to 
increased learning time for students that remained in classrooms, and decreased time and effort 
expended by teachers and administrative staff in addressing the disciplinary problems of students 
removed from classrooms. 
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For the llrst-through-third evaluation years, students at Technology Immersion schools had 
significantly lower school attendance rates than control students — however, in the fourth year, 
differences between the attendance rates of treatment and control students were smaller and 
statistically nonsignificant. Previous studies of technology projects have linked one-to-one 
computing with fewer school absences and late arrivals compared to non-laptop students (e.g., 
Stevenson, 1999). Across the first three evaluation years, in contrast to previous studies, our research 
has shown that students at Technology Immersion schools attended school less regularly than control 
students. For example, economically advantaged Cohort 2 (seventh graders) had an average attendance 
rate of 96.9% compared to 97.2% for control students, and economically disadvantaged immersion 
students, similarly, had significantly lower attendance rates than con t red -group students (95.9% versus 
96.3%). In the fourth year, economically advantaged and disadvantaged Cohort 2 students at 
immersion schools (who were now eighth graders) continued to have slightly lower attendance rates 
(96.6% and 95.4%, respectively) than their control-group counterparts (97.0% and 95.8%, 
respectively), but the school attendance rate differences between groups were extremely small and not 
statistically significant. Likewise, the attendance-rate differences for Cohort 3 treatment and control 
students (seventh graders) were very small and statistically insignificant in the fourth year. 

In previous years, we conjectured that the lower school attendance rates of immersion students might 
reflect the tendency for some students to occasionally skip school in order to use their laptops at home. 
In the fourth year, as noted previously, some control students also received individual laptops (about 
9% and 17% of Cohorts 2 and 3 control students, respectively). Thus, it is feasible that the 
introduction of laptops in control schools had a slightly negative effect on those students’ school 
attendance rates (similar to the lower attendance of immersion students). In any case, as detailed in the 
section below, the modestly lower average school attendance rates of immersion students across years 
have not been associated with lower academic achievement. 

Effects of Technology Immersion on Academic Achievement 

Increasing middle-school students’ academic achievement in core subjects as measured by state 
assessments was the ultimate goal of Technology Immersion. For analyses reported below, students’ 
Texas Assessment of Knowledge and Skills (TAKS) scale scores were standardized as T scores with a 
mean of 50 and a standard deviation of 10. Analyses for Cohort 2 (eighth graders) included about 
1,570 immersion and 1,700 control students; Cohort 3 (seventh graders) included about 1,690 
immersion and 1,970 control students. We also investigated the TAKS performance of Cohort 1 
students (ninth graders) who attended Technology Immersion and control schools from sixth-through- 
eighth grade and then enrolled in traditional high schools, which typically did not provide individual 
laptops for students, through TAKS testing in spring 2008. 

Longitudinal data across multiple student cohorts has allowed researchers to examine the replicability 
of achievement effects. Given that small effects are noteworthy when effects arc replicated (e.g., Abelson, 
1985; Cohen, 1994; Schmidt, 1996), we have reported some effects as statistically significant using a less 
stringent criterion {p - < .10) if findings provided evidence substantiating important trends. Students 
completed TAKS tests annually in reading and mathematics, so the evidence of immersion effects is 
stronger for those subject areas. In contrast, evidence for science, social studies, and writing is limited 
because students’ completed those assessments at periodic intervals. 

Table 8.1 summarizes the estimated magnitude of the Technology Immersion effect on TAKS reading and 
mathematics achievement across student cohorts. Estimated effects arc described as the cumulative 
growth in T-score units for Technology Immersion and control groups, the mean cumulative growth 
differences between groups in T-score units, and the estimated sizes of the effects in standard deviation 
units. Major findings on the effects of Technology Immersion on TAKS achievement follow the table. 
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Table 8.1. HLM Model-Based Estimations of Technology Immersion Effects 
on TAKS scores by Subject, Economic Disadvantage Status, and Student Cohort 

| Cumulative Growth 



Immersion 

Control 

Mean 

Standard 


'/’-score 

'/-score 

'/’-Score 

Deviation 

Assessment/Student Cohort 

Growth 

Growth 

Difference 

Units 

TAKS Reading, Advantaged 





Cohort 1 : 9th graders, post-immersion 

0.76 

-0.06 

0.8 l f 

.08 

Cohort 2: 8th graders, 3 immersion years 

1.10 

0.39 

0.70 

.07 

Cohort 3: 7th graders, 2 immersion years 

0.00 

-0.21 

0.21 

.02 

TAKS Reading, Disadvantaged 





Cohort 1 : 9th graders, post-immersion 

2.22 

1.41 

0.8 1 + 

.08 

Cohort 2: 8th graders, three immersion years 

2.02 

1.32 

0.70 

.07 

Cohort 3: 7th graders, two immersion years 

0.22 

0.01 

0.21 

.02 

TAKS Mathematics, Advantaged 





Cohort 1: 9th graders, post-immersion 

2.14 

0.88 

1.25 

.13 

Cohort 2: 8th graders, 3 immersion years 

1.27 

-0.69 

1.96* 

.20 

Cohort 3: 7th graders, 2 immersion years 

0.74 

-0.83 

1.57* 

.16 

TAKS Mathematics, Disadvantaged 





Cohort 1: 9th graders, post-immersion 

1.63 

0.38 

1.25 

.13 

Cohort 2: 8th graders, three immersion years 

1.81 

-0.15 

1.96* 

.20 

Cohort 3: 7th graders, two immersion years 

0.09 

-1.48 

1.57* 

.16 


Note. Estimated T-score growth for students attending schools with average levels of poverty. Cumulative growth 
in T-score units (mean= 50, standard deviation = 10). Standard deviation units = T - score difference/10. 

V < -10. *p < .05. 


Technology Immersion had no statistically significant effect on TAKS reading achievement for 
Cohort 2 (eighth graders) or Cohort 3 (seventh graders) — however, for Cohort 1 (ninth 
graders), there was a marginally significant and positive sustaining effect of Technology 
Immersion on students’ TAK reading scores. After controlling for student and school poverty, there 
were no statistically significant effects of immersion on the TAKS reading growth rates for either 
Cohort 2 (eighth graders) or Cohort 3 (seventh graders). The immersion effects were positive but not 
by statistically significant margins. For Cohort 1 (ninth graders) there was a statistically significant 
and positive sustaining effect of immersion on the TAKS reading growth rates of students who had 
attended immersion middle schools and then moved on to mainly traditional high schools (p < .06). 
The reading achievement of post-immersion students increased by 0.19 T-score point per year, 
whereas the achievement of control ninth graders decreased by about 0.01 T-score point per year. 
Across Cohorts 1 and 2, economically disadvantaged students grew in reading achievement at 
significantly faster rates than their more affluent peers (0.56 and 0.67 T- score points per year for 
immersion students, respectively; 0.35 and 0.44 T-scorc points for control-group students, 
respectively). For TAKS reading, the sizes of immersion effects in standard deviation units (.08, .07, 
and .02) were very small but increased with longer exposure to Technology Immersion and through 
the post-immersion year in high school. 

Technology Immersion had a statistically significant effect on TAKS mathematics achievement 
for Cohort 2 (eighth graders) and Cohort 3 (seventh graders). For Cohort 1 (ninth graders), the 
sustaining effect of immersion on TAKS mathematics scores was positive but not by a 
statistically significant margin. After controlling for student and school poverty, Technology 
Immersion had a statistically significant effect on students’ growth rates for TAKS mathematics 
(p < .05) for Cohorts 2 and 3 students. Estimated yearly TAKS mathematics growth rates for 
economically advantaged students in immersion schools (0.42 and 0.37 T-score points per year for 
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Cohorts 2 and 3, respectively) significantly outpaced their control-group counterparts (-0.23 and -0.42 
T-score points, respectively). Similarly, estimated yearly TAKS mathematics growth rates for 
economically disadvantaged students in immersion schools (0.60 and 0.05 T-score points per year for 
Cohorts 2 and 3, respectively) were significantly more positive their control-group counterparts (-0.05 
and -0.74 T-score points, respectively). There were no statistically significant differences between the 
TAKS mathematics outcomes for Cohort 1 post-immersion and control-group ninth graders. Still, the 
TAKS mathematics growth rates of economically advantaged and disadvantaged post-immersion ninth 
graders (0.53 and 0.41 T-score points per year, respectively) were steeper than the rates of their 
control-group counterparts (0.22 and 0.10 T-score points per year). For TAKS mathematics, the sizes 
of immersion effects in standard deviation units for Cohort 2 (.20) and Cohort 3 (.16) were small but 
statistically significant. The estimated immersion effect for Cohort 1, ninth graders in standard 
deviation units (0. 13) was similar to the magnitude of the effect detected at the end of their eighth- 
grade year. 

Similar to the previous year, students’ use of their laptops for Home Learning — a measure of the 
extent to which a student used a laptop outside of school for homework in the four core-subject 
areas or for learning games — was the strongest implementation predictor of students’ TAKS 
reading and mathematics scores. Given that the level of implementation of Technology Immersion 
varied from school to school, classroom to classroom, and student to student, we used a series of 
hierarchical linear models to investigate the relationships between implementation levels and student 
academic achievement. Specifically, Student Access and Use was an aggregate implementation 
measure of the extent to which a student had access to a laptop throughout the school year (number of 
days), the frequency of technology use for learning in core-subject classes, and the extent of laptop use 
for homework and learning games. Student-level HLM results showed that the composite measure of 
Student Access and Use was a consistently positive although not always statistically significant 
predictor of students’ TAKS reading and mathematics scores for Cohorts 2 and 3. Of the three 
elements of Student Access and Use, students’ use of their laptops for Home Learning — a measure of 
students’ use of laptops outside of school for homework in core-subject areas and for learning 
games — was the strongest predictor of both TAKS reading and mathematics achievement across both 
cohorts. 

For Cohort 2 (eighth graders), the extent of Home Learning was a positive but nonstatistically 
significant predictor of students’ TAKS reading achievement and a positive and marginally significant 
predictor of TAKS mathematics achievement. For Cohort 3 (seventh graders), the extent of laptop use 
for Home Learning was a positive and statistically significant predictor of both TAKS reading and 
mathematics scores. In contrast, reading and mathematics teachers’ reported levels of Classroom 
Immersion were inconsistent predictors of students’ TAKS scores. 

The findings for Home Learning underscore the important role that individual student laptops play in 
promoting ubiquitous learning and in equalizing the out-of-school learning opportunities for students 
in disadvantaged family and school situations (Burbules, 2007; Dede, 2007). Individual student 
laptops, in contrast to laptops on carts or computers available in libraries, labs, and classrooms, expand 
where and how student learning occurs. In a third-year implementation study of the traits of higher 
Technology Immersion schools and teachers, researchers found that students at higher Technology 
Immersion schools typically had access to laptops “24/7.” Teachers at higher immersion schools 
encouraged students’ use of laptops outside of school by engaging students in projects or assignments 
that motivated students to continue working outside of class. Also, access to electronic textbooks on 
laptops motivated many students to continue working on chapter assignments outside of school 
(Shapley et al., 2008). 
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Conclusions about the effects of Technology Immersion on TAKS social studies and science 
scores remain in doubt. However, outcomes for TAKS writing, which involved the 
administration of the TAKS assessment in traditional paper-and-pencil format, have 
consistently favored control students although not by statistically significant margins. Since 
TAKS tests for social studies, science, and writing arc not administered annually, immersion effects 
for those subject areas cannot be replicated across cohorts and years. Accordingly, it is difficult to 
draw definitive conclusions about the effects of Technology Immersion for these subject areas. 
Available results have revealed no statistically significant differences between treatment and control 
groups for TAKS social studies, science, or writing scores. Treatment-control group differences for 
TAKS writing, however, have consistently favored students at control schools. 

Social studies. The TAKS social studies test is administered for the first time in 8th grade, so 
students’ 5th grade TAKS reading scores were used to adjust for prior achievement. After controlling 
for Cohort 2 eighth graders’ reading achievement, demographic characteristics, and school poverty, 
there was no statistically significant effect of immersion on students’ 2008 TAKS social studies 
scores. The immersion effect was virtually zero (0.006 T-score point). 

Science. After controlling for prior achievement (5th grade science score), demographic 
characteristics, and school poverty, there was no statistically significant effect of immersion on 
Cohort 2 eighth graders’ TAKS science achievement. The immersion effect was positive (0.48 T-score 
point) but not by a statistically significant margin. 

Writing. After controlling for Cohort 3, seventh graders’ pretest writing scores (4th grade writing 
score), demographic characteristics, and campus poverty, there was no statistically significant effect of 
immersion on students’ 2008 TAKS writing scores. Similar to previous years, the immersion effect 
was negative (-0.73 T-score point). Across evaluation years, seventh graders in immersion schools, on 
average, have had consistently lower TAKS writing scores (-0.91, -0.28, and -0.73 T-score points for 
Cohorts 1, 2, and 3 seventh graders, respectively). It is possible that the administration of the TAKS 
assessment in paper-and-pencil format may underestimate the writing performance of Technology 
Immersion students who have used word processing software on a regular basis for written 
schoolwork. Some research studies have shown that traditional assessments underestimate the writing 
performance of students who are accustomed to using word processors for writing and are not allowed 
to use word processors when tested (Russell & Haney, 1997; Russell & Plati, 2001). 

Nature of Fourth-Year Implementation 

The section below describes the progress made by schools in implementing the Technology 
Immersion model across the second through fourth project years. 

The overall level of implementation of the Technology Immersion model increased to some 
extent across years — even so, just a quarter of schools reached substantial levels of immersion 
by the end of the fourth implementation year. Full implementation of the Technology Immersion 
model requires support in several ways: Leadership, Teacher Support (buy-in), Parent and Community 
Support, Technical Support, and Professional Development. Given adequate supports, teachers are 
expected to reach high levels of Classroom Immersion, and Student Access and Use of technology is 
expected to be robust. Mean immersion standard scores revealed small yearly increases across most of 
the implementation support components (Leadership, Teacher Support, Parent/Community Support, 
and Professional Development) as well as increases in teachers’ overall level of Classroom Immersion. 
In contrast, the level of Student Access and Use declined across years. Mean fourth-year immersion 
standard scores (ranging from 2.69 to 3.19 on a 4-point implementation scale) showed that many 
schools needed stronger supports, especially in the areas of parent and community support for 
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technology use, technical supports that addressed obstacles to technology use, and professional 
development for teachers. 

Core-subject teachers at the majority of schools reported only partial levels of Classroom 
Immersion in the fourth year. Teachers’ mean scores at a fifth of schools, however, revealed 
substantial levels of Classroom Immersion. As a whole, the standards-based implementation scores 
for Classroom Immersion increased slightly across years (from 2.48 to 2.69 on a 4-point scale). 
Standard scores for four of the five elements of Classroom Immersion showed somewhat stronger 
implementation in the fourth year, with the largest increase for teachers’ use of technology for their 
own puiposes (Professional Productivity) and the smallest change for classroom integration 
(Technology Integration). The frequency with which teachers allowed students in their classrooms to 
use technology for learning activities (Student Activities) remained relatively stable across years. 

Students’ access to and use of laptops for learning within and outside of school continued to fall 
well short of expectations in the fourth year. The percentage of schools with at least partial levels 
of Student Access and Use decreased across three evaluation years (76%, 68%, and 57%), while the 
percentage of schools with minimal student access and use increased (24%, 32%, and 43%). Several 
factors affected students’ opportunities to use their laptops for learning both within classrooms and 
outside of school. These factors mainly included time lost for repairs due to aging laptops, schools that 
opted to transfer laptops from individual students to carts or classroom sets, schools that restricted 
students’ use of laptops outside of school, and teachers’ preferences regarding classroom laptop use. 
Year-to-year comparisons showed that the mean implementation level for Laptop Access Days 
increased between the third- and-fourth implementation years (from 2.50 to 2.64 on a 4-point scale) 
due to more consistent student “access” to laptops (although not “ownership”) at some schools on 
carts or as classroom sets. At the same time, the yearly mean implementation levels for laptop use for 
Core-Content Learning (classroom laptop use) decreased across years (2.07, 2.12, and 1.95) and the 
use of laptops for Home Learning, likewise, decreased over time (1.75, 1.84, and 1.63). This trend is 
consistent with what other researchers have documented. When teachers are the “gatekeepers” of 
students’ technology use, many teachers, especially veterans, will opt to continue traditional practices 
and reject practices that require innovation and instructional change (Cuban, 2002; Russell, Bebell, & 
Higgins, 2004). 

Implementation and Sustainability 

Implementation Fidelity of the Technology Immersion Model 

During spring 2008 site visits at schools, researchers asked principals, technology specialists, and 
teachers to describe their progress in implementing Technology Immersion, and in retrospect, what 
they would have done differently to improve implementation. Key findings from interviews and focus 
groups are summarized below. 

Nearly all of the Technology Immersion Pilot (TIP) grantees said the lack of a start-up year for 
planning was a major barrier to effective implementation of Technology Immersion. The 

majority of middle schools received their TIP grant award just before the start of the first project year. 
Thus, many thought implementation would have progressed more smoothly if there had been a start- 
up year to plan for immersion. Various respondents said a planning year would have allowed them to 
(a) have conversations with teachers about the decision to become an “immersed school,” (b) develop 
a plan for managing laptops (especially at larger campuses with as many as 1,500 laptops), (c) build 
the school’s infrastructure for wireless technology, (d) have teachers become more accustomed to 
laptops and available software and digital resources, (e) provide professional development for teachers 
to strengthen their technical skills and ability to plan technology-integrated lessons, and (e) give 
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teachers a chance to “try out” lessons with laptops in the classroom before students had their own 
laptops. One administrator said, “We flew by the seat of our pants. . .learning after the fact.” Other 
respondents said the lack of planning made the first implementation year “hectic” or “stressful.” 
Several grantees, however, described practical lessons that could only be learned through experiences 
with one-to-one computing. 

TIP grantees who were more successful thought that committed leaders, thorough planning, 
teacher buy-in, preliminary professional development for teachers, and a commitment to the 
transformation of student learning were keys to their successful implementation of Technology 
Immersion. Respondents at middle schools that had been more successful attributed effective 
implementation to several factors. Foremost, despite a quick start, district and school administrators 
had a well-conceived plan for implementation, were excited about the project, and listened to teacher 
input. Administrators had “high expectations” for technology use but allowed time for teachers to 
become comfortable. One teacher explained: 

We had the right combination of encouragement and push. . .Leadership, encouragement, and 
push. It wasn’t punitive, it was positive. . .but they kept up the pressure. . .That constant, 
positive pressure moved me forward. 

Professional development for teachers was a high priority. Training typically began before the first 
year started and was ongoing across implementation years. These schools also had collegial cultures. 
Teachers learned by “seeing what other teachers were doing and how they were implementing 
technology.” “We were all in this together,” explained one teacher, “Some teachers liked what they 
had always done, but we were willing and ready to try.” The improvement of students’ learning 
experiences was a driving force for higher quality implementation at these schools. Despite myriad 
laptop management issues, respondents believed the challenges had been worthwhile because one-to- 
one student laptops and digital resources had increased the depth of learning across subject areas, 
exposed students to more real-life experiences, and allowed students to demonstrate greater 
responsibility. 

Many TIP grantees reported that administrative turnover, noncommittal teachers, insufficient 
professional development, inadequate school infrastructures, and laptop management problems 
were impediments to effective implementation of the Technology Immersion model. Respondents 
at many schools cited obstacles that had derailed their implementation efforts. At many schools, 
constant principal turnover caused major set-backs each year and undermined teacher buy-in for 
immersion. Many teachers expressed noncommittal attitudes about the continuation of Technology 
Immersion at their schools, which seemed to stem from four main sources: (a) frustrations caused by 
the concurrent distribution of laptops to teachers and students in the first year, (b) the insufficiency of 
their preparation to meet technical demands and manage technology-integrated lessons, (c) students’ 
inconsistent access to laptops for classroom activities, and (d) uncertainty about their students’ 
capacity to handle one-to-one laptop access (i.e., students were too young or immature, lacked 
sufficient technical and keyboarding skills, had insufficient prior experience with computers, behaved 
irresponsibly with expensive laptops, or wanted to use technology to “play” rather than “learn”). One 
administrator summed it up by saying, the “success of Technology Immersion depends on the 
teacher — some arc hesitant.” Many teachers wished that professional development had been provided 
earlier, and that the training received had focused on content-specific lesson plans. Many teachers new 
to schools felt unprepared to deal with laptops in classrooms. Additionally, respondents at these 
schools often cited problems with inconsistent wireless Internet services, insufficient technical staff to 
deal with laptop repairs in a timely manner, and students who did not bring their laptops to school or 
class regularly. 
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A qualitative report — Third-Year (2006-07) Traits of Higher Technology Immersion Schools and 
Teachers — provides a comprehensive examination of implementation successes and challenges at 
Technology Immersion schools (Shapley et al., 2008). 

Sustainability of the Technology Immersion Model 

As part of site visits, administrators, technology specialists, and teachers also commented on 
sustainability of the Technology Immersion model at their schools. Key findings are summarized 
below. 

Sustainability depended on the commitment of district leaders to Technology Immersion and to 
long-range planning for continuation. The principals and technology specialists at many campuses 
had not been directly involved in planning for the sustainability of Technology Immersion beyond the 
fourth year, and in fact, most said that decisions about continuation would rest with district 
administrators. In other cases, plans were in place to continue Technology Immersion at middle 
schools, and some districts were planning to expand one-to-one computing to high schools or upper 
elementary grades. Respondents who described explicit plans for continuation cited the key role of the 
superintendent and board of trustees. “If your district and school board are committed to it, it is 
sustainable,” said one respondent. Sustainability of Technology Immersion rested on planning ahead 
and being prepared for future years, including actions such as (a) having a plan for the replacement of 
worn and outdated laptops (b) allocating resources to support continuous teacher professional 
development, and (c) allocating resources for technical support and student Help Desk facilities. Some 
administrators said a plan for a three-year replacement cycle for laptops was essential. 

Sustainability of Technology Immersion depended on the adequacy of funds to support 
continuation. With grant funds ending, many campuses were uncertain about how Technology 
Immersion could be sustained financially. Given limited local and state dollars for technology, most 
respondents described their hopes for winning additional grant awards to continue their one-to-one 
laptop programs. For example, some administrators hoped to receive funding from the Bill & Melinda 
Gates Foundation, Vision 20/20 grants, STAR grants, or U.S. Department of Agriculture funds for 
rural school districts. Many principals were optimistic about their chances of securing grants to 
support continuation but had doubts about receiving financial support from their districts or the state. 

A few districts and schools, however, had used local funds to support Technology Immersion. For 
example, one district decided to eliminate computer labs and apply the money toward individual 
laptops, some districts were considering lease/purchase options for laptops, one school planned to use 
their district technology allotment to purchase laptops, and other schools were planning to use 
technology funds to purchase laptop parts and supplies to keep worn laptops up and running for 
another school year. A charter school had a generous business partner, an enthusiastic supporter of the 
Technology Immersion concept, who was studying how one-to-one laptop access for students could be 
continued at more modest costs. An administrator in one district said continuation of Technology 
Immersion depended on a local bond issue. If the bond failed, there would be no money for laptops at 
the school. In this district, budget shortfalls were causing cuts to administrative and teaching positions, 
so there was little hope of receiving district money for technology. 

A superintendent who was interviewed explained the importance of having dedicated local funds to 
sustain a one-to-one project: “It started with the TIP grant, but it is a vision of ours, we are committed 
to it. We committed to it locally and committed a ton of resources outside of the grant.” Although this 
district was trying to use local funds to support the project, the superintendent believed the state 
should provide more or more flexible financial support. In particular, state funding allocations 
earmarked by lawmakers for specific programs prevented local education agencies from combining 
state and local funds for school-reform initiatives, which local educators believed had greater potential 
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for improving students’ academic performance. Additionally, several respondents thought the state 
should provide additional technology funding so that schools did not have to depend entirely on local 
funds. Having sufficient local funds for technology was an acute problem for a property-poor district 
that depended on state funds and grants to purchase technology, but frequently did not qualify for 
grants because the district maintained high TAKS test scores. One respondent said the continuation of 
Technology Immersion simply “depended on how much money the state legislature makes available to 
schools.” A few administrators believed traditional paper textbooks are outmoded and state funds 
invested in printing and delivering millions of textbooks across the state should be used to fund 
technology. 

For other campuses, it seemed that the TIP project was just another grant program, and once funding 
ended, the TIP project would disappear - . One principal said, “Unless we come up with another source 
of funding for equipment and software, technology will be cut.” Other principals said they would 
“love to continue immersion” but saw no way to financially sustain the current model. One principal 
said, Technology Immersion is only sustainable in an “ideal world.” Another administrator said, “If 
the grant is not renewed, it would be the end of one-to-one computing.” 

Sustainability of Technology Immersion was associated with educators’ beliefs about 
technology’s value for addressing the learning styles and needs of students, and educators’ 
commitment to move toward digital school environments. School leaders who wanted to continue 
one-to-one laptop projects often linked their intentions with hopes for student learning. Administrators 
cited goals that involved moving students “away from drill and practice” and toward “creation of 
products;” preparing students for the 21st century by building literacy, problem solving, and 
collaborative skills; expanding learning outside of school; exposing students to “worldwide cultures” 
so they have a sense of being part of a larger community; and making learning “more than 
regurgitating information back on a test.” In describing the major accomplishments of Technology 
Immersion, a committed administrator said: 

We have impacted a ton of kids in a positive way. It has impacted their learning; it has 
impacted their exposure to the world; it has impacted their education experience; it has 
impacted them personally, and by that I mean their self-esteem, the way they feel about 
themselves and the way they feel about education; it has made a huge difference. 

Other campuses were committed to the continuation of Technology Immersion because 
superintendents saw the value of going “paperless.” This involved purchasing electronic versions of 
textbooks (on CDs or online) instead of traditional paper copies, and conducting student assessments 
online. Some spoke of the value of online college coursework for students, virtual learning 
opportunities, and reduced costs for small, rural school districts through shared teachers for 
coursework delivered via videoconferencing. Some administrators said they simply could not 
“imagine being without laptops.” They had seen such growth in the use of software applications for 
purposes such as TAKS preparation that it would be difficult to be without laptops. “We would be 
stepping back in time,” said one administrator who believed laptops played a critical role in preparing 
students for college where they “will be required to do everything with technology.” 

Some school administrators were committed to continuation of Technology Immersion, but they 
wondered if an incremental approach to implementation might have improved their long-term 
prospects for sustainability. Some principals, especially those at larger schools, believed it might be 
easier to move toward full implementation of the Technology Immersion model by introducing student 
laptops gradually, immersing one grade at a time. One principal said, “I would phase it in grade by 
grade, so that it would be done by groups of teachers and students.” Another respondent argued for a 
three-year immersion cycle for middle-school grades, with the cycle tied to the replacement of worn 
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laptops. Some teachers thought a gradual approach to introducing laptops during the school year 
would help. For example, laptops could initially be used as class sets (i.e., for the first six weeks) until 
students became acquainted with laptops and the guidelines for appropriate care. Nevertheless, one 
administrator explained why the ultimate goal should be school-wide implementation of the 
Technology Immersion model: 

The full immersion model was the best way to go. . .because we are all on the same page. It is 
a campus initiative. So the conversations are not just horizontal, it is vertical as well. That’s 
the power of it. . .And the electives, it is across the board a whole-campus initiative. 

At the end of TIP grants, several schools that had experienced great difficulty implementing the 
Technology Immersion model were planning to abandon one-to-one student laptop access and 
return to more conventional configurations of educational technology. Some schools that had 
experienced severe problems implementing the Technology Immersion model were considering other 
options to continue student access to technology at their schools. Several respondents described these 
kinds of changes: (a) one-to-one computing would be sustained only at selected grade levels, 

(b) student access to laptops would be restricted to in-school use only, (c) laptops would be distributed 
as classroom sets, or (d) laptops would be placed on mobile carts for teacher checkout. A number of 
teachers expressed preferences for having classroom sets of laptops instead of individual student 
laptops. These teachers believed classroom sets would minimize laptop “wear and tear” and also 
“ensure that all students have a laptop” in class. Although some principals thought the TIP project had 
been successful, changes reflected concerns about the adequacy of financial and personnel resources to 
sustain one-to-one computing at their schools. Decisions to move toward more traditional technology 
configurations were typically intended to prolong the life of laptops. 

Findings from four evaluation years suggest that Technology Immersion can be implemented 
and is sustainable if districts and schools are committed to the model — however, other 
approaches to technology use may be appropriate for some districts and schools. Over four years, 
it became evident that Technology Immersion involved more than just buying laptops for students. 
Technology Immersion is a comprehensive model for transforming the school culture, and the nature 
of teaching and learning, and expanding the educational boundaries of the school. This study has 
shown that fundamental school change is difficult and requires a long-term commitment at all levels of 
the school system (board members, superintendent, principals, teachers, students, and parents). Given 
the challenges of implementing and sustaining the Technology Immersion model, statewide 
implementation may not be possible. However, those districts and schools that arc committed to 
Technology Immersion should have state support for their innovative school-reform efforts; at the 
same time, other districts and schools should receive support for alternative technology-based 
initiatives that have research-based evidence of effectiveness. 
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Appendix A 

Theoretical Framework for Technology Immersion — Literature Review 


The theoretical framework (Figure 1.1) guides the evaluation. The research literature underpinning the 
Technology Immersion model and the theoretical framework is provided in sections to follow. In some 
cases, sources relate specifically to educational technology, whereas in other instances, evidence 
comes from studies of education in general. Research evidence for some variables is relatively robust; 
in other areas, evidence is weaker. Although research on one-to-one computing initiatives has grown 
in recent years, there arc still few experimental studies or studies with well matched comparison 
groups that provide evidence of causal effects. 

Technology Immersion Model 

The Technology Immersion model assumes that effective technology use in schools and classrooms 
requires robust technology access, technical and pedagogical support for implementation, professional 
development for educators in using technology effectively, and readily available curricular and 
assessment resources that support the state’s foundation curriculum (English language aits, 
mathematics, science, and social studies). 

First, technology use in schools and classrooms requires robust access. Despite school-level 
improvements in the ratio of students to instructional computers in Texas (Education Week, 2007), 
recent survey data show that an average of 2.9 or less classroom computers is insufficient to allow 
every student access (Shapley, Benner, Fleikes, & Pieper, 2002; Shapley et al., 2006). In response to 
prevailing conditions, Technology Immersion calls for one-to-one student access to computers. The 
Texas project, in contrast to one-to-one laptop initiatives being implemented in other states and school 
districts (e.g., Maine, Michigan, New Hampshire, Vermont, Henrico County in Virginia) adopts a 
comprehensive approach. In particular, Technology Immersion assumes that increased access to and 
use of technology in schools requires adequate technical and pedagogical support. Schools must have 
robust electronic networks to support wireless laptops and digital content. Campus-based support is 
also vital, as ample studies show the importance of on-site support personnel who assist teachers in 
learning to use technology, troubleshooting technical problems, and effectively integrating technology 
into lessons (e.g.. National Center for Education Statistics [NCES], 2000; Ringstaff & Kelley, 2002; 
Ronnkvist, Dexter, & Anderson, 2000; Shapley et al., 2002). 

In addition, the Technology Immersion model assumes that teachers must have effective professional 
development. High-quality professional development, as research demonstrates, is of longer duration 
and provides richer learning experiences, more comprehensive investigation of topics, and time for 
practice and experimentation (e.g.. Claret, Porter, Desimone, Birman, & Yoon, 2001; Lawless & 
Pellegrino, 2007; Penuel, Fishman, Yamaguchi, & Gallagher, 2007; Smerdon, et al., 2000). Moreover, 
when a particular technology is mastered over time, it is more likely to be incorporated into instruction 
(Zhao & Frank, 2003). Teachers also need follow-up support as they acquire and implement new skills 
in the instructional setting (Bradburn & Osborne, 2007; Garet et al., 2001; Nugent & Fox, 2007; Sulla, 
1999). Professional development should also focus on subject-specific content or specific teaching 
methods. For technology, this means building teachers’ basic technology skills as well as their 
understanding of curricular integration (CEO Forum, 2000, 2001; Denton, Davis, & Strader, 2001; 
Ringstaff & Kelly, 2002; Web-Based Education Commission, 2000). The alignment of professional 
development activities with teachers’ personal goals for learning is also important in advancing 
teacher change (Garet et al., 2001; Penuel et al., 2007). 
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Additionally, technology-related professional development should be paid of broader professional 
growth initiatives in schools (Fullan & Hargreaves, 1996; Mann, Shakeshaft, Becker, & Kottkamp, 
1999; Newmann & Associates, 1996). Professional development activities that include collective 
participation (e.g., whole schools or teachers of the same subjects or grades) are more likely to be 
coherent with teachers’ experiences and needs (Caret et al., 2001). A leadership development 
component is crucial because research points consistently to the important role of school leaders in 
successful implementation of technology (Bradburn & Osborne, 2007; Johnston & Cooley, 2001; 
Pitler, 2005). 

Technology Immersion also requires curricular and assessment resources that support the state’s 
curriculum. Thus, laptops in immersion schools include software that allows students and educators to 
use wireless laptops as a tool for teaching, learning, communication, and productivity. Digital 
resources (e.g., online, CD-ROMS, stored on local networks) also provide students with a means for 
more personalized learning activities, and interactive technologies allow them to build new knowledge 
by doing, receiving feedback, and refining their understanding. Technologies also help students to 
acquire more information, visualize difficult-to-understand concepts, and advance understanding 
(Bransford, Brown, & Cocking, 2003). Online formative assessments enable teachers to diagnose 
students’ strengths and needs or to assess their mastery of curricular standards. 

Theoretical Framework: School-Level Variables 

In a “technology immersed” school, technology resources are ingrained in the school’s organizational 
and cultural environment. Technology immersion, therefore, should change not just classroom 
instruction and learning, but also the nature of interactions between student and teacher, teacher and 
teacher, teacher and principal, and the school within the surrounding community (Dwyer, 1994). 
Considering the systemic nature of technology immersion, the evaluation examines factors that help to 
explain how and under what conditions technology affects students’ learning opportunities and 
academic achievement. The sections below describe the key variables of interest at the school level, 
including leadership, innovative culture, parent and community support, and technical support. 

Leadership 

Over the past several decades, researchers have concluded consistently that school leadership is 
critical in developing and maintaining conditions that support school change and academic 
improvement (e.g., Hallinger & Heck, 1996 cited in Spillane, 2003; Leithwood, Seashore, Anderson, 

& Wahl strom, 2004). Similarly, administrative support is a major factor that influences technology 
integration (International Society for Technology in Education, 2002; Bradburn & Osborne, 2007). 
Leaders in a technology-enhanced environment must be “champions of technology, teaching, learning, 
and students” (Johnston and Cooley, 2000, p. 95). The principal, in particular, is a pivotal figure in 
effective technology implementation. The visionary principal is one who sees the integral relationship 
between technology and education, and marshals resources to help teachers master effective practices 
(Tinucci, 2000). Additionally, effective principals are “transformational leaders” who create more 
collaborative teaching and learning environments through their facilitation of opportunities for 
technology specialists and teachers to share their knowledge, experiences, and insights (Bradburn & 
Osborne, 2007). 

A consistent vision and plan for change is also essential for whole-school reform efforts such as 
technology immersion. Shared vision, or buy-in, moves schools toward substantive changes in 
instructional approaches and improved student outcomes (Leithwood et al., 2004). Conversely, 
without broad-based support, technology immersion may be untapped resource that has little impact 
on student learning (Cradler, 1992; Means & Olson, 1994). 
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Innovative Culture 

The school culture may either promote or impede whole-school initiatives such as technology 
immersion. When undertaking innovation, the organization’s shared commitment to change and ability 
to build capacity for doing things in a new way are important (Senge, 1999). In education, some 
schools are more successful than others in enacting and sustaining innovation, and in more effective 
schools, changed practice is a collective rather than an individual enterprise (Fullan, 1993). Similarly, 
movement towards new ways of teaching and learning with technology is more significant if teachers 
are able to work collaboratively (Chapman, 1996). Shared professional learning opportunities provide 
a viable means to stimulate innovative teaching practices (Birman, Desimone, Porter, & Garet, 2000; 
Dibbon, 2003). Considering prior research, we believe that educators’ collective experiences at 
immersion campuses will advance their shared understanding of technology’s use and encourage 
integration efforts. Schools that begin the project with more collaborative cultures may advance at a 
faster pace (Fullan, 1999). 

Parent and Community Support 

The local community also may influence technology immersion. Its constituents consist of parents, 
neighborhood residents, local professionals, and elected school board officials. Educating and 
involving the community has been identified as a key component in ensuring successful change in 
educational practices (Desimone, 2002; Goertz, Floden, & O’Day, 1996; Leithwood et al., 2004). If 
parents and community members are “on the same page” as the school with regard to technology 
immersion, they can contribute the kind of supports and resources required for changes in educational 
practices. At immersion campuses, community outreach may take many forms, such as participation 
on a technology committee, attendance at informational sessions or workshops, the dissemination of 
information through district and campus websites, or media releases to spread the word about 
technology immersion. Most important, in a one-to-one computing project, parents must be partners in 
assuming responsibility for the appropriate use of laptops outside of the school. 

Technical Support 

Texas has strongly supported the infusion of technology into its schools (Texas Education Agency, 
2002; 2006). Consequently, at the start of this project, both treatment and control campuses had 
existing inventories of technology hardware, software, and educational programs. Districts and 
campuses also had human resources such as technology coordinators and technical support personnel 
who supported technology at the district and campus levels. Given existing contextual conditions, and 
the infusion of resources through technology immersion, an examination of the nature and quality of 
technical support at participating schools is important. 

Theoretical Framework:Teacher Variables 

At the teacher level, we theorize that technology immersion leads to increased technology proficiency, 
greater use of technology for professional productivity, more frequent opportunities for students to use 
technology in classrooms, and pedagogical changes such as increased technology integration and more 
learner-centered instruction. New technology also is expected to advance the intellectual demands of 
lessons and assignments. Moreover, teachers in schools that are immersed in technology should begin 
to collaborate more often with their peers as they experiment with new instructional technologies and 
digital resources. 
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Technology Proficiency 


A number of studies associate teachers’ technology proficiencies with technology implementation. 
Research indicates that teachers need a solid foundation of technology literacy before they can 
successfully integrate technology into the curriculum. Teachers must learn to use technology 
comfortably and efficiently (Dusick, 1998-1999; Goldsworthy, 2000). Studies also show that teachers 
with stronger computer skills use technology in a greater number of ways and on a more regular basis, 
and these teachers are more likely to increase their technology-use frequency over time (Ronnkvist, 
Dexter, & Anderson, 2000). Moreover, teachers with the strongest technology proficiencies use 
technology in more innovative ways in their content areas (Becker, 2000). 

Unfortunately, research indicates that many teachers lack the proficiencies and understanding 
necessary to apply technology resources to instruction and learning effectively. A national study found 
that more than half of teachers felt only somewhat prepared to use technology for instruction, and 
more experienced teachers felt less prepared than their more novice counterparts (Smerdon et al., 
2000). Surveys of Texas teachers have revealed improvements in proficiencies across time, but 
teachers’ proficiency levels remained below targeted standards (Shapley, Benner, Heikes, & Pieper, 
2002). Similarly, 2005-06 statewide outcomes for the Texas Teacher STaR Chart (a measure of 
teachers’ technology readiness) showed that nearly three of four Texas teachers rated their progress 
relative to the Teaching and Learning area as either Early Tech (14.7%) or Developing Tech (55.6%). 
Only one in four teachers believed they had attained proficiencies designated as Advanced Tech 
(23.7%) or Targeted Tech (5.8%) (Texas Region 10 Education Service Center & Texas Education 
Agency, 2006). 

Professional Productivity 

Skilled teachers also are more likely to use technology as a tool to enhance their own professional 
productivity, including actions such as communicating with students and parents by email, creating 
electronic lesson plans, or accessing information from the Internet for lessons (Shapley et al., 2002). 
Researchers typically have not investigated teachers' use of technology for professional productivity, 
but it is important in Texas because state standards call for teachers to use technology for 
communicating effectively, as well as for acquiring, analyzing, and evaluating a variety of electronic 
information. In an immersed school, teachers are expected to increasingly communicate by email, 
report attendance and submit lesson plans electronically, post information on a class or campus 
website, and analyze and interpret electronic data from assessments. 

Classroom Technology Use 

The link between increased technology access and increased classroom use is well documented. 
Teachers use computers and the Internet more often when technologies are available in their 
classrooms rather than in other locations in the school (Becker, 2001; Smerdon et al., 2000). Teachers 
involved in Maine’s one-to-one initiative, in fact, used technology more often, possessed a broad 
knowledge of technology resources, and made progress in incorporating technology into practice 
(MEPRI, 2004). Thus, we assume that providing laptops for each student in an immersed school will 
increase students’ opportunities for classroom technology use. 

Technology Integration and Learner-Centered Instruction 

Abundant technology hardware and software is important, but if those resources are not well 
integrated into instructional approaches and learning experiences, the impact on student achievement 
may be negligible. Notably, studies show that teachers’ ideologies affect the likelihood of technology 
integration, with teachers’ perceived costs and benefits influencing changed practices (Zhao & Frank, 
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2003). Research also suggests that teachers’ understanding of new learning theories and understanding 
of how technology supports enriched learning opportunities are important (Bransford, Brown, & 
Cocking, 2003; Johnston & Cooley, 2001). Researchers studying the Apple Classrooms of Tomorrow 
(ACOT) found that abundant access to classroom technology changed teachers’ beliefs as well as their 
instructional approach. Teachers’ beliefs and practices evolved along a technology integration 
continuum that gradually led to effective instructional practices. Movement from the entry phase to 
invention (technology-intensive environments) required time and ongoing support (Dwyer, Ringstaff, 
& Sandholtz, 1991). 

Specifically, researchers found that ACOT teachers began to incorporate more collaborative work and 
fewer teacher-centered, lecture-oriented lessons in favor of student-centered ones (Baker, Gearhart, & 
Herman, 1994). Subsequent studies, likewise, have found evidence of teachers adjusting their 
pedagogical style, with students taking more responsibility for their own learning in one-to-one laptop 
classrooms (MEPRI, 2003), and classroom structures that shifted from large group to students working 
independently or to more student-centered activities (Rockman ET AL., 1998; Russell, Bebell, Cowan, 
& Corbelli, 2002). Other evidence, however, suggests that some teachers view technology as an add- 
on or reward for students who finish their seatwork rather than an integral part of their pedagogical 
repertoire (Rockman ET AL., 1998). 

Intellectual Challenge 

Technology immersion’s main benefit may stem from opportunities for more complex modes of 
teaching and learning. Research on technology-infused classrooms reveals positive attributes, such as 
the ability to bring real-life problems into the classroom or high-quality simulations of them. 
Technology also allows teachers to model thinking strategies and allows individual learners to 
approach tasks in different ways using different learning strategics (Goldman, Cole, & Syer, 1999; 
Many, Fyfe, Lewis, & Mitchell, 1996; Sulla, 1999; Temple & Rodero, 1995). This view of 
technology’s potential for more advanced learning contrasts with evidence on prevailing classroom 
conditions. While three-quarters of teachers nationally report using computers or the Internet for 
instruction, most lessons fail to involve complex inquiries, explorations, or problem-solving activities 
(Doherty & Orlofsky, 2001). Similarly, Texas students and teachers use technology mainly at a basic 
level, with technology used most often for tasks such as conducting Internet research on an assigned 
topic (Shapley et al., 2002). 

Collaboration 

Research suggests that teachers need time to discuss technology use with other teachers. Professional 
collaboration includes communicating with educators in similar situations and with teachers who have 
previous technology experiences. Collaboration may occur in face-to-face meetings or through 
technology venues such as email or videoconferencing. Teachers in the Maine laptop initiative, for 
example, believed their most effective professional development activity was informal help from 
colleagues. E-mail, listservs, and websites enabled Maine teachers to exchange information and stay in 
touch with their peers (MEPRI, 2003). Moreover, Zhao and Frank report that “teachers who perceived 
pressure from colleagues were more likely to use computers for their own puiposes, and teachers who 
received help from colleagues were more likely to use computers with their students” (2003, p. 825). 

Theoretical Framework: Student Variables 

Over the past decade, a growing body of research points to positive effects of technology on students’ 
skills, learning, and achievement. In the research literature, evidence suggests that technology access 
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fosters positive student effects for technology use, technical proficiencies, motivation and engagement, 
intellectually challenging schoolwork, self-direction, and to a lesser extent, academic achievement. 

Technology Use 

Technology is used more often for instructional and learning puiposes in one-to-one laptop classrooms 
(Russell, Bebell, & Higgins, n.d.). Additionally, students involved in ubiquitous technology initiatives 
use technology more often outside of school. Russell et al. (n.d.) found that students in one-to-one 
classrooms used computers at home more frequently for academic puiposes. Likewise, other 
researchers found that students spent less time watching television and more time on homework after 
they received laptop computers (Baldwin, 1999). Moreover, laptops provided a means of “closing the 
digital divide” between more advantaged students who had access to computers and the Internet at 
home and those without technology outside of school (Rockman, 2003). 

Technology Proficiency 

Students’ technology proficiencies reportedly increase with ubiquitous technology. Laptop students in 
one study considered themselves more proficient users of Word, Excel, PowerPoint, the Internet, 
email, and CD-ROMS than non-laptop students (Rockman ET AL., 1998). Similarly, fifth and sixth 
graders who received laptop computers in another study reported increased computer skills and better 
Internet research capabilities (Lowther, Ross, & Morrison, 2001). In another study, German high 
school students with laptops made greater gains than comparison students on measures of technology 
literacy, such as knowledge of hardware and the operating system, productivity tools, and Internet use 
(Schaumburg, 2001). 

Motivation and Engagement 

Numerous studies report links between one-to-one technology and increased student engagement 
(MEPRI, 2003; Rockman ET AL., 1998; Russell et al., n.d.; Woodul, Vitale, & Scott, 2000). The five- 
year ACOT evaluation established a link between technology use and student attitudes. Students 
voluntarily used time outside of school to work on technology-based projects, and they often initiated 
their own computer-related projects (Baker, Gearhart, & Herman, 1994). Students involved in the 
Maine Learning Technology Initiative, similarly, found school and learning more interesting and 
preferred using laptops for most school-related tasks (MEPRI, 2003). 

Additionally, studies have examined the relationship between technology and student behavior. In a 
statewide study in Florida, middle schools experienced fewer student conduct violations and 
disciplinary actions as the number of computers in use per student increased (Barron, Hogarty, 
Kromery, & Lenkway, 1999). Other studies, likewise, report decreased discipline problems associated 
with one-to-one computing (Baldwin, 1999; MEPRI, 2003). In another study, a computerized 
curriculum positively affected the psychosocial and academic outcomes of students identified as 
chronically disruptive (Aeby, Powell, & Carpenter-Achy, 1999-2000). 

An evaluation of the North Carolina Laptop Notebook Project revealed a strong correlation between 
computer use and improved school attendance. Students participating in the laptop program had fewer 
absences and late arrivals as compared to non-participants (Stevenson, 1998). In Henrico County 
Public Schools in Virginia, preliminary evidence linked increased student motivation, engagement, 
and interest to one-to-one computing (Zucker & McGee, 2005). 
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Intellectual Work 


Existing studies suggest that student technology use most commonly involves productivity tools, 
Internet research, and drill and practice activities. Activities involving higher-order thinking and peer 
collaboration, such as technology-based projects, multimedia authoring, problem solving with 
spreadsheets or databases, or correspondence with experts, are less common (Becker, 1999, 2001; 
Denton, Davis, & Strader, 2001; Smerdon et al., 2000). Contrary to prevalent practice, some believe 
that technology, at its best, can “facilitate deep exploration and integration of information, high-level 
thinking, and profound engagement by allowing students to design, explore, experiment, access 
information, and model complex phenomena” (Goldman et al., 1999). Additionally, technology allows 
students increased access to and use of a wide range of information, facilitating greater inquiry and 
investigation, exposure to places and resources beyond the classroom, and development of a stronger 
knowledge base (CEO Forum, 2001; Johnston & Cooley, 2001). 

New circumstances and opportunities — not technology on its own — can impact student achievement. 
Several studies have established tentative links between interactive technologies and higher level 
reasoning and problem solving (Baker et al., 1994; Hopson, Simms, & Knezek, 2002). New 
technologies, apparently, allow students to build knowledge by doing, receiving feedback, and 
continually refining their understanding (Barron et al., 1999; Bereiter & Scardamalia, 1993). 
Technology also provides a medium for bringing real-world problems into the classroom for students 
to explore and solve. Students involved in the Jasper Woodbury Problem Solving Series, for example, 
had positive gains in mathematical problem solving, communication abilities, and attitudes toward 
mathematics (e.g., Cognition and Technology Group at Vanderbilt, 1997). 

Self-Directed Learning 

Several studies associate technology use with increased student self-directed learning. The connection 
assumes that working one-to-one with technology allows students to have hands-on, self-directed 
experiences since they work independently much of the time. The theory of self-regulation posits that 
a learner who knows how to be self-directed and independent will be more successful than one who is 
highly dependent on structured guidance (Zimmerman, 1989). The teacher’s role is to scaffold 
learning by making thinking processes more tangible and by modeling learning strategies (Bolhuis, 
1996; Corno, 1992; Leal, 1993). Since self-directed learners are responsible owners and managers of 
their own learning process, control shifts over time from teachers to learners (Garrison, 1997). 

Self-regulated or self-directed strategics enable learners to solve problems in new domains (Ertmer & 
Newby, 1996; Morrow, Sharkey, & Firestone, 1993) or to solve real-world problems (Bolhuis, 1996; 
Temple & Rodero, 1995). For example, in computer-supported science classes, middle-school students 
took more responsibility for their learning, and concurrently, displayed greater competence in complex 
problem-solving strategics (Raghavan, Sartoris, & Glaser, 1997). Another study suggested that 
students who learned in a self-directed environment were more productive. When writers were 
allowed to choose their own topics, they wrote more often and they wrote longer pieces (Morrow et 
al., 1993). 

Academic Achievement 

The ultimate goal of technology immersion is increasing the academic progress of students. Available 
evidence on the effects of laptops on student achievement comes from a few studies that have made 
comparisons between student groups with and without technology. Findings, although limited, have 
generally been positive. 
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The strongest evidence on the effects of laptops on achievement is in the area of writing. Lowther, 
Ross, and Morrison (2001, 2003) reported highly significant effects favoring sixth- and seventh-grade 
students with laptops over control students for dimensions of writing, such as ideas and content, 
organization, and style. In a less methodologically rigorous study, Rockman ET AL. (1999) found that 
laptop students outscored non-laptop students on four measures of writing, including content; 
organization; language, voice, and style; and mechanics, conventions, and presentation. 

Some studies also have reported positive effects of one-to-one laptop access on students higher order 
problem solving (Lowther et al., 2003). Evaluation of a laptop project in Beaufort County, West 
Virginia, which focused on outcomes measured by a nationally standardized achievement test, found 
that laptop students participating in the program for two years had higher language, reading, and 
mathematics scores than non-laptop students (Stevenson, 1998). However, since there was no 
statistical control for prior achievement, findings are in doubt. Certainly, additional research studies 
with experimental designs are needed to draw definitive conclusions about the effects of one-to-one 
initiatives on student achievement. 
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Appendix B 

Characteristics of Participating Schools 


The schools participating in the study arc compared in Table B.l. The distribution of middle 
schools across campus and district enrollment categories shows the comparability of treatment 
and control groups. For both groups, middle schools are typically small (enrolling 600 students or 
less), and they are located either in small or very small districts (enrolling 2,999 students or less) 
or large districts (enrolling 10,000 students or more). 


Table B.l. Campus and District Enrollment by Comparison Group 



Immersion N= 2 1 

Control N= 21 

Number of students 

Number 

Percent 

Number 

Percent 

Campus 

300 or less 

12 

57.1 

12 

57.1 

301-600 

5 

23.8 

4 

19.0 

601 or more 

4 

19.0 

5 

23.8 

District ! 

999 or less 

8 

38.1 

8 

38.1 

1,000-2,999 

6 

28.6 

5 

23.8 

3,000-9,999 

0 

0.0 

0 

0.0 

10,000 or more 

7 

33.3 

8 

38.1 


Note. Two campuses (one experimental and one control) were excluded from the 
comparison groups in the second year. 


Tables B.2 and B.3 provide campus-level data for each of the 42 schools included in the study. 
Again, data show that the treatment and control schools are reasonably well matched on baseline 
characteristics. Middle schools are highly concentrated in rural and very small districts across the 
state. Still, over a third of districts and schools arc in large cities or suburban locations in or 
around cities. The sample also includes campus charter schools (one each for the treatment and 
control group) located in a major urban district. 
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Table B.2. Characteristics of Technology Immersion and Matched Control Schools 
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Table B.3. Characteristics of Technology Immersion and Matched Control Schools ( Continued ) 



•X 

O 

c 


bO 

c 3 


g 

<D 

U 


g 

-O 


G 

3 

o 


T 3 

c 

3 

3 


T 3 .a 
c is 

O C/3 
8 ^ 


D 


rL ^ 

&-S 

o o 

feh ^ 

<D 3 


3 

■§ 


a s 


<D 


> o 
rS 
*0 .S 

3 2 
O o 

° '£ 
W C /3 

3 ' ~ 
O ^ 


*3 ,£2 


D 


'S § 

Ph X) 
X 3 
<D 3 

s 

O 3 ^ 


<D . . 
V 3 0 ) 

2 Q- 
Ph 

S H 


£ § 

H S 

a o 
5 u 


■c 


bO 

c 

£ 

o 


T 3 

s 

<3 

O. 

<L> 

C 


3 

X 


T 3 

C 

3 

O 


T 3 

3 

3 


•c 


c 

c 

X 


§ 

■e 

•§ 


3 

X 

a> 

H 


3 

X 


C/3 

S 

s 

3 

o 

Z 


o 

CN 


3 

S-H 

X 

o 


0) 

> 

3 

X 

T 3 

3 

3 


O 

bQ 


3 

3 

X 


• G 


o 

o 

X 


3 

IX 


o 

h 


3 

O 

Z 

o 

o 

o 

© 

o 


o 

o 

o 


<N 

X 


o 

o 

G 

<3 

X 


-T 3 

G 

3 

O 

O 


G 

0) 


X 

? 

o 

feb 

<D 


<D 

> 

3 

X 

-a 

c 

<3 


o 

bo 


3 

<3 

X 

5-1 


•J3 


o 

o 

X 


111 





Appendix C 

Survey Items and Scale Reliabilities 


Table C.l. Items and Reliabilities for School-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring. Spring Spring 
2005 2006 2007 

Spring 

2008 

Leadership and System Support 

0.91 

0.92 

0.94 

0.97 

0.94 

The principal consults with staff before making decisions about instructional technology that affect 
us. 






In this school, there are clear expectations that technology will be used to enhance student 
learning. 






The principal in my school actively encourages teachers to pursue professional development 
geared towards curricular integration of technology. 






Our school has a well-developed technology plan that guides all technology integration efforts. 






The principal is an effective leader for instructional technology in this school. 






Overall, considering the uses of technology in my school today, 1 am confident that this use is 
leading to increased student achievement. 






The principal encourages teachers to be innovative and try new methods. 






The principal is willing to support through funding or manpower teachers' efforts at technology 
integration. 






Administrators in this school help teachers to use technology to access, analyze, and interpret 
student performance data. 






Teachers receive adequate administrative support to integrate technology into classroom practice. 






Teachers and administrators rely on research-proven teaching and learning principles in making 
decisions about technology use. 






When our school has professional development focused on technology, the principal often 
participates. 






Classroom Technology Integration 

0.67 

0.68 

0.76 

0.75 

0.71 

Students have adequate access to technology resources in my classroom (e.g., digital cameras, 
scanners, projectors). 






1 incorporate the TEKS for student technology applications into my content-area lessons. 






1 have received sufficient training to incorporate technology into my instruction. 






1 use technology to assess student performance and plan instruction. 






Technical Support 

0.71 

0.71 

0.66 

0.67 

0.68 

Most of our school computers are kept in good working condition. 






Internet connections in my class are often too slow or not working. 






My requests for technical assistance are addressed in a timely manner. 






Materials (e.g., software, printer supplies) for classroom use of computers are readily available in 
my school. 






Problems such as computers freezing or an inability to access the Internet make it difficult for me 
to use technology. 






Innovative Culture 

0.78 

0.79 

0.82 

0.82 

0.82 

Teachers in this school share an understanding about how technology will be used to enhance 
learning. 






Teachers in this school are continually learning and seeking new ideas. 






Teachers are not afraid to learn about new technologies and use them with their class(es). 






Teachers in this school are generally supportive of technology integration efforts. 






Parent and Community Support 

0.78 

0.79 

0.85 

0.84 

0.84 

Parents support our school's emphasis on technology. 






The surrounding community actively supports our instructional efforts with technology. 
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Table C.2 Items and Reliabilities for Teacher-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Spring Spring 
2005 2006 2007 

Spring 

2008 

Technology Proficiency: 1 am confident that 1 can. . . 

0.97 

0.97 

0.97 

0.97 

0.96 

Send email to coworkers, parents, or peers. 






Collaborate through subscribing to a discussion list. 






Create an address book to send email to several people at once. 






Send a document as an attachment to an email message. 






Use a variety of search strategies, including key word and Boolean logic to find Web pages 
related to my subject matter interests. 






Search for and find a Web site with information about the Alamo. 






Create my own World Wide Web home page. 






Keep track of Web sites 1 have visited so that 1 can return to them later. (An example is using 
bookmarks.) 






Find primary sources of information on the Internet that 1 can use in my teaching. 






Use a spreadsheet (e.g., excel) to enter and calculate numbers. 






Use a spreadsheet to create a pie chart. 






Create a newsletter using desktop publishing techniques, including graphics & text in 3 columns. 






Perform basic software application functions such as opening an application program and 
creating, modifying, printing, and saving documents. 






Plan, create, and edit documents using word processing software (e.g., Word). 






Use the computer to create a slideshow presentation (e.g., PowerPoint). 






Plan, create, and edit databases using database software (e.g., Access). 






Use a database to search for and sort information and create reports. 






Use graphic organizers and/or systems thinking software (Inspiration, Stella, etc.) to teach 
concepts. 






Use drawing or painting software (e.g., Paint, Illustrator) to create pictures. 






Create a lesson or unit that incorporates subject matter software as an integral part. 






Use technology to collaborate with other colleagues who are distant from my classroom. 






Describe 5 software programs that 1 would select and use in my teaching. 






Write a plan with a budget to buy technology for my classroom. 






Teach my students about copyright issues as they relate to the Internet including citing sources. 






Take photos with a digital camera, save in a digitized format, and use in an electronic document. 






Scan images from a print source such as a book, save them in a digitized format, and use them 
in an electronic document. 






Create products incorporating text, audio, video, and graphics using multimedia authoring 
programs (e.g., Authorware, HyperStudio). 






Professional Productivity: As a teacher, 1... 

0.93 

0.94 

0.94 

0.91 

0.92 

Keep administrative records (e.g., attendance). 






Manage student assessment data (e.g., electronic gradebooks). 






Use technology to analyze and interpret student data to guide instruction. 






Create electronic lesson plans. 






Communicate with students. 






Communicate with parents. 






Communicate with colleagues/other professionals. 






Create instructional materials (e.g., tests, handouts). 






Gather information from the internet to create a lesson (e.g., text, video, clipart). 






Access model lesson plans integrating technology. 






Deliver information using presentation software (e.g., PowerPoint). 






Deliver information using multimedia presentations (text, audio, video, grgraphics). 






Post homework, class requirements, or project information on a website. 






Administer a formative assessment using Texas Mathematics Diagnostic System. 






Administer other online assessments. 






Use the internet at home for instructional purposes. 






Use a computer to do schoolwork at home. 






Students' Technology Use: Students in my class use technology to... 

0.95 

0.98 

0.98 

0.96 

0.97 

Express themselves in writing (e.g., word processing). 






Learn and practice skills (e.g., instructional software or educational games). 






Enter, calculate, and graph information (e.g., Excel spreadsheet). 






Create a database of information for a class project (e.g., Filemaker Pro, Access). 






Create and make presentations (e.g., PowerPoint). 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Spring Spring 
2005 2006 2007 

Spring 

2008 

Communicate by email with peers, experts, or others on topics they are studying. 






Use online discussions to gather information for an assignment (e.g., through discussion boards 
or videoconferencing). 






Conduct internet research on an assigned topic. 






Conduct multimedia research (reference CDs, online encyclopedias). 






Enhance or express conceptual understanding through simulation/modeling software. 






Visually represent or investigate concepts (e.g., through concept mapping, graphing, reading 
charts). 






Produce print products (e.g., desktop publishing). 






Produce multimedia reports/projects (e.g., with video, graphics, and sound editing). 






Analyze information using tools such as graphing calculators or digital microscopes. 






Design web sites or web pages. 






Complete a test or quiz (e.g., online assessments, Texas Math Diagnostic System). 






Other (specify) 






Collaboration: As a teacher, 1... 

0.90 

0.92 

0.93 

0.92 

0.92 

Act as a coach or mentor to other teachers or staff at my school. (May include teaching in- 
service workshop in your school.) 






Receive coaching or mentoring from an external (non-school) source such as a professional 
curriculum developer. 






Receive coaching or mentoring from an internal source, such as another teacher or technology 
coordinator. 






Have informal discussions with colleagues regarding strategies for integrating technology. 






Receive feedback from other teachers based on their observations of my teaching. 






Provide feedback to other teachers based on my observations of their teaching. 






Consult with other teachers about certain students' technology skills or use. 






Exchange feedback with other teachers based on student work that used technology. 






Work with a subject-area peer to develop a lesson plan or class activity using technology. 






Work with a colleague in a different subject area to develop a lesson plan. 






Participate in a study group with other teachers on a technology-related topic. 






Technology Integration 

0.94 

0.95 

0.95 

0.91 

0.91 

1 alter my instructional use of the classroom computer(s) based upon the newest software 
applications and research on teaching, learning, and standards-based curriculum. 






My students discover innovative ways to use classroom computers to make a difference in their 
lives. 






1 allocate time for students to practice their computer skills on the classroom computer(s). 






1 integrate the most current research on teaching and learning when using the classroom 
computer(s). 






In my classroom, students use technology-based computer and Internet resources beyond the 
school (NASA, other government agencies, private sector) to solve authentic problems. 






My students’ authentic problem solving is supported by continuous access to a vast array of 
computer-based tools and technology. 






1 plan computer-related activities in my classroom that will improve my students' basic skills 
(e.g., reading, writing, math computation). 






It is easy for me to design student-centered, integrated curriculum units that use the classroom 
computer(s) in a seamless fashion. 






1 seek out activities that promote increased problem-solving and critical thinking using the 
classroom computer(s). 






Using cutting edge technology and computers, 1 have stretched the instructional computing in 
my classroom. 






Learner-Centered Instruction 

0.75 

0.80 

0.81 

0.81 

0.83 

Students' authentic use of information and inquiry skills guides the type of instructional materials 
used in my classroom. 






My students are involved in establishing individual goals within the classroom curriculum. 






In addition to traditional assessments, 1 consistently provide alternative assessment 
opportunities that encourage students to “showcase" their content understanding in 
nontraditional ways. 






My instructional approach emphasizes experiential learning, student involvement, and students 
solving “real-world” issues. 






Resistance to Integration 

0.70 

0.72 

0.77 

0.81 

0.83 

1 do not find computers to be a necessary part of classroom instruction. 






Using the classroom computer(s) is not a priority for me this school year. 






1 do not find the use of computers to be practical for my students. 
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Table C.3. Items and Reliabilities for Student-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Spring Spring 
2005 2006 2007 

Spring 

2008 

Technology Proficiency: How far along are you in learning to... 

0.94 

0.94 

0.94 

0.94 

0.94 

open, create, modify, print, and save documents 






use a digital camera and/or scanner to get pictures into the computer 






send a document as an attachment to an email 






keep track of Web sites 1 have visited so that 1 can return to them later (using bookmarks, etc.) 






enter information on the computer using proper keyboarding skills 






gather information from CD-ROMS 






use online reference databases (online encyclopedias, newspapers, Library of Congress, etc.) 
to gather information 






use a search engine to find information about a topic (Alamo, etc.) on the Web 






narrow Web searches using key words and Boolean logic (such as “or,” “and,” or “not") 






use online discussions with experts or mentors to gather information 






evaluate information found on the Web for accuracy 






use a word processor (AppleWorks, Word, etc.) to write and print a story or report 






use a spreadsheet (AppleWorks, Excel, etc.) to enter and calculate numbers 






use a spreadsheet to create graphs 






use a database (AppleWorks, Access, etc.) to enter information 






use a database to search for and sort information and create reports 






use software (Keynote, PowerPoint, etc.) to create a presentation 






use drawing or painting software (Paint, Illustrator, etc.) to create pictures 






use a video camera to make a video 






use software (HyperStudio, Authorware, etc.) to create a multimedia product 






use email to send and receive messages 






use software (Frontpage, Publisher, etc.) to create web pages 






Technology Use in School: In your English language arts, mathematics, social studies, and 
science classes, how often do your teachers have you... 

0.90 

0.92 

0.91 

0.92 

0.93 

use a word processor (AppleWorks, Word, etc.) to write a story or report. 






use software to learn and practice skills (Riverdeep, Compass Learning, PLATO Learning, etc.). 






use a spreadsheet (Excel, etc.) to enter and calculate numbers or create graphs for an 
assignment. 






create a database of information (Filemaker Pro, Access, etc.) for a class project. 






create a presentation (PowerPoint, etc.) and present information to classmates or others. 






communicate by email with friends, experts, and others about topics you are studying. 






use online discussions to gather information for an assignment (discussion boards, 
videoconferencing, etc.). 






conduct Internet research on an assigned topic. 






use tools, such as graphing calculators or digital microscopes, to analyze information. 






produce print products (with desktop publishing software). 






create multimedia reports or projects (with video, graphics, and sound editing). 






use technology to complete a test or quiz. 






Other 






Technical Problems 

0.83 

0.85 

0.84 

0.77 

0.77 

The computer is broken or slow. 






The program 1 need is not on the computer. 






The Internet connection is too slow or not working. 






A website 1 need is blocked by a filter. 






Sharing a computer makes it hard to finish assignments. 






My teacher can’t fix things when something goes wrong. 






Other (describe) 






Small-Group Work: When students work together in small groups in my classes, we... 

0.80 

0.83 

0.83 

0.83 

0.83 

review and give advice on each other's work. 






tutor or coach each other on difficult work. 






make a presentation for the rest of the class. 






brainstorm solutions to problems. 






discuss previous class assignments. 






produce a report or project. 






School Satisfaction 

0.77 

0.82 

0.80 

0.80 

0.81 

1 am satisfied with the work that 1 do in my classes. 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Spring Spring 
2005 2006 2007 

Spring 

2008 

1 understand why 1 am doing the things we do in my classes. 






The things we do in my classes will help me as an adult. 






The work we do in my classes will be useful to me in the job 1 hope to have as an adult. 






1 work hard in my classes because the work is meaningful. 






What 1 learn in my classes is more important than the grade 1 receive. 






Self-Directed Learning 

0.88 

0.89 

0.89 

0.89 

0.88 

If I'm confused in class, 1 ask the teacher or another student for help. 






Sometimes, if 1 think an assignment is too tough, 1 purposely don't try hard. Then if 1 don't do 
well, 1 don't feel bad. 






At the end of a project or assignment, I'll think about how hard 1 worked and whether 1 would do 
anything differently next time. 






It's important to me that 1 understand my schoolwork really well. 






Even when 1 think my schoolwork is boring, 1 keep working until I'm finished. 






Before 1 begin studying, 1 think about or list the things I'm going to do during my study time. 






Even when I'm supposed to learn about something boring, 1 keep working until 1 finish. 






When my teacher writes comments on assignments, 1 don't read them unless 1 have to. 






When we start a new unit, 1 like to know what we're going to be learning and how I'll know if I've 
learned it well. 






When the teacher calls on me, and 1 make a mistake in class, 1 can honestly say that 1 don't feel 
bad. 






When 1 do well on a big project, it's because I've worked hard. 






1 work harder than 1 need to on my schoolwork, because that's just the way 1 am. 






I'll recopy my notes or make diagrams of what we're learning to try and remember it better. 






1 don't like asking for help with my schoolwork. 






If a topic is too hard, it's really hard for me to stay motivated. 






If 1 know I'm going to do badly on a task, 1 try to avoid it, even if 1 know I'd learn a lot from it. 






There are some subjects I'm just bad at. 






A lot of times, I'll wait until the last minute to do my homework or study for a test. 






1 know 1 can make a schedule to get my work done on time and stick to it. 






When I'm doing homework, 1 rush to finish if 1 have ,a friend coming over or if a good TV show 
is about to start. 






I'll look through mistakes 1 made on earlier assignments so 1 don't make the same mistakes on 
new assignments. 






When I'm done writing a report, 1 read it over carefully and think 0 about whether I've done a 
good job. 






Even if 1 try, 1 can't make myself concentrate on schoolwork when there are more interesting 
things to do. 






When I’m reading a chapter, 1 ask myself questions to make sure 1 understand the material. 






There are some subjects 1 just can’t understand, even if 1 try hard. 






When 1 get a bad grade, 1 feel dumb. 






I'll pick a tough project where 1 would learn a lot over an easy project, even if it means I'll have to 
work harder to get a good grade 






This happens to me a lot: I'll study for a test and think 1 understand everything; then 1 take the 
test and don't do very well. 






1 don't really take notes when I'm reading something for school. 






When 1 get a grade 1 don't like, I'll spend time trying to figure out what 1 could have done 
differently. 






When 1 do badly on a project, 1 feel okay as long as 1 did better than some of the other kids in 
my class. 






When 1 answer a question wrong in class, 1 end up wishing I'd never spoken up. 






When 1 get a bad grade, it's because 1 could have studied more or because 1 should have done 
something differently, like taking better notes. 






If I'm having trouble concentrating, 1 find a place to study where 1 won't be distracted. 






The things we're learning in my class are usually really interesting. 






If 1 have to choose, I'd rather get good grades in a class than learn a lot. 






When a big project or report is assigned, 1 make a mental or written schedule to make sure 
everything gets done on time. 






I'll usually ask someone (like my parents, friends or teacher) to give me feedback on my ideas 
when I'm working on a big assignment. 






1 know from past experience exactly what 1 have to do (like schedule a certain amount of time, 
or take notes in a particular way) if 1 want to do well on my schoolwork. 






If an assignment isn't going to count toward my grade, 1 don't need to know how well 1 did on it. 







117 




Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Spring Spring 
2005 2006 2007 

Spring 

2008 

1 only feel bad about a low grade if 1 think 1 didn't work hard enough, or if 1 think 1 made careless 
mistakes 






When 1 read, 1 put the important ideas into my own words. 






When I'm not feeling motivated, 1 can't, make myself study. 






When 1 don't understand things in class, 1 end up thinking it's because I'm not that smart. 






When we have a reading assignment, I'll read through it one time, but 1 don't really go back 
through it to check how well 1 remember it. 






1 know 1 can do well in school if 1 try hard enough. 






1 don't ask for help, even if 1 don't understand the directions for an assignment. 






1 wouldn't do any homework if 1 didn't have to. 
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Appendix D 

Measurement of Implementation Fidelity 


Defining Technology Immersion 

The Texas Education Agency selected three lead vendors as providers of technology immersion 
packages (Dell Computer, Inc., Apple Computer Inc., and Region 1 Education Service Center [ESC]). 
Sections to follow provide descriptions of the components of technology immersion packages. 

Wireless Laptops and Productivity Software 

All vendors offered a wireless laptop as the mobile computing device. Campuses could select either 
Apple laptops (iBook and MAC OSX) or Dell laptops (Inspiron or Latitude with Windows OS). For 
Apple laptops, AppleWorks provides a suite of productivity tools, including Keynote presentation 
software, Internet Explorer, Apple Mail, iCal calendars, iChat instant messaging, and iLife Digital 
Media Suite (iMovie, iPhoto, iTunes, GarageBand, and iDVD). For Dell laptops, Microsoft Office 
includes Word, Excel, Outlook, PowerPoint, and Access. In addition, eChcilk serves as a “portal” to 
other web-based applications and resources included in the immersion package and a student-safe 
email solution. Region 1 ESC provided Dell products. 

Online Instructional and Assessment Resources 

Immersion packages included a variety of digital resources. Apple included the following online 
resources: netTrekker (an academic Internet search engine). Beyond Books from Apex Learning 
(reading, science, and social studies online), ClcissTools Math from Apex Learning (complete math 
instruction), ExploreLearning Math and Science (supplemental math/science curriculum), 
TeenBizSOOO from Achieve 3000 (differentiated reading instruction), and My Access Writing from 
Vantage Learning (support for writing proficiency). Dell, Inc. selected netTrekker (an academic 
Internet search engine) and Connected Tech from Classroom Connect (technology-based lessons and 
projects). Region 1 ESC selected Connected Tech but also added a variety of teaching and learning 
resources including Unitedstreaming (digital videos), Encyclopedia Britannica, EBSCO (databases), 
NewsBank , and K12 Teaching and Learning Center. For the Apple package, AssessmentMaster 
(Renaissance Learning) provides a formative assessment in all four core subject areas. Both the Dell 
and Region 1 ESC packages provide i-Know (CTB McGraw Hill) for core-subject assessment. In 
addition, all campuses have access to the online Texas Mathematics Diagnostic System (TMDS) and 
Texas Science Diagnostic System (TMDS) that are provided free of charge by the state. 

Professional Development 

Each immersion package includes a different professional development provider. Apple uses its own 
professional development model, whereas the Dell package relies on Pearson Achievement Solutions, 
a commercial provider (formerly Co-nect), to support professional development. Region 1 ESC uses a 
combination of service center support plus other services offered through Connected Coaching and 
Connected University. Although the professional development models and providers differ, they all 
were expected to include some common required elements, such as support for immersion package 
components, the design of technology-enhanced learning environments and experiences, lesson 
development in the core-subject areas, sustained learning opportunities, and ongoing coaching and 
support. Individual districts and campuses collaborated with vendors to develop specific professional 
development plans for their teachers and other staff. 
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Technical and Pedagogical Support 

Each technology immersion package provider also is required to provide campus-based technical 
support to advance the effective use of technology for teaching and learning. Apple designed a Master 
Service and Support Program. Dell established a Call Center dedicated to technical support for TIP 
grantees as well as an 800 telephone number for hardware and software support. Region 1 ESC had an 
online and telephone HelpDesk to answer questions and provide assistance. 

In sum, the RFQ process created technology immersion packages with common elements. Still, the 
complexity and variability of the treatment makes it critically important for researchers to document 
not only how and how well technology immersion is implemented but also to identify factors that 
contribute to implementation variations. 

Measuring Implementation 

In the second through fourth years, we employed a two-part approach to the measurement of 
implementation fidelity. First, we used indicators to describe each campus’ progress on a 4-step scale 
toward immersion standards. Rating scales for components and related elements identified four levels 
of immersion: minimal (0 to 1.99), partial (2.00 to 2.99), substantial (3.00 to 3.49), and full (3.50 to 
4.00). Second, we used quantitative implementation indices that gauged the level of technology 
immersion using standardized scores (z scores). Both the immersion standard scores and 
implementation indices were derived from values for seven components: (a) Leadership, (b) Teacher 
Support, (c) Parent and Community Support, (d) Technical Support, (e) Professional Development, 

(f) Classroom Immersion, and (g) Student Access and Use. The following sections describe the seven 
components of technology immersion and related measurement procedures. Table D.l shows the 
scoring rubrics for immersion indicators, and Table D.2 describes the data sources used to generate 
scores. 

Supports for Implementation 

Leadership. Our measure of administrative leadership comes from teacher survey items (12) that 
yield a Leadership scale score. Items assess the extent to which administrators involved staff in 
decisions, set clear expectations for technology use, encourage and participate in professional 
development, have a well-developed technology plan, promote teacher innovation, and provide 
necessary resources and administrative support. Teachers rated the extent of their agreement on a 5- 
point scale ranging from 0 ( strongly disagree ) to 4 ( strongly agree). To achieve substantial to full 
immersion, teachers had to agree or strongly agree that administrators provided technology 
leadership. A Leadership Index was generated by transforming the scale score to a z score. 

Teacher Support. Although implementation may be affected by the characteristics of individual 
teachers, it also may reflect the collective disposition of teachers toward the adoption of new and 
innovative practices. Our measure of teacher commitment to technology immersion comes from 
teacher survey items (4) measuring a Teacher Support scale (i.e., Innovative Culture). Items gauged 
the extent to which teachers in the school share an understanding about technology use for student 
learning, arc continually learning and seeking new ideas, are not afraid to learn about and use new 
technologies, and are generally supportive of technology integration efforts. Teachers rated the extent 
of their agreement on a 5-point scale ranging from 0 ( strongly disagree) to 4 ( strongly agree), with 
substantial to full immersion tied to the strength of teacher agreement. A Teacher Support Index was 
generated by transforming the scale score to a z score. 

Parent and Community Support. Support from parents and community members is also a key paid 
of implementation because they must understand the goals of technology immersion, assume 
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responsibility along with their children, and assist in enacting effective policies. Our measure of Parent 
and Community Support is a scale score composed of teacher survey items (2). These items indicate 
the extent to which parents support the school’s emphasis on technology and the community actively 
supports instructional efforts with technology. Teachers rated the extent of their agreement on a 5- 
point scale ranging from 0 ( strongly disagree ) to 4 ( strongly agree). Substantial to full immersion 
reflected the strength of teacher agreement. A Parent/Community Support Index was generated by 
transforming the scale score to a z score. 

Technical Support. On a fully immersed campus, sufficient technical support and a healthy 
infrastructure are expected to alleviate technical problems that might interfere with the use of 
technology in the classroom, school, and beyond. Our measure for technical support comes from 
teacher survey items (5) contributing to a Technical Support scale score. Teachers indicated the extent 
of their agreement on a 5-point scale ranging from 0 ( strongly disagree ) to 4 ( strongly agree) that 
computers are kept in good working order, requests for assistance are addressed in a timely way, 
Internet connections work adequately, and classroom materials are readily available. A Technical 
Support Index was generated by transforming the scale score to a z score. 

Professional Development. In constructing measures of professional development, we drew from 
research conducted on the effectiveness of the Eisenhower Professional Development Program (e.g., 
Garet, Porter, Desimone, Birman, & Yoon, 2001). Key features of quality professional development 
provided a framework for examining dimensions of schools’ and vendors’ professional development 
models. Data for measures come from core-subject teachers’ responses to survey items. 

First, we measured the total number of Contact Hours that core-subject teachers spent in technology- 
related professional development during the past school year. In addition, professional development 
models for technology immersion were required to include a classroom support component, so we 
measured Classroom Support as the extent to which core teachers indicated that they received 
modeling, coaching or mentoring from an internal source (such as another teacher or technology 
coordinator), or an external source (such a professional curriculum developer). Teachers rated the 
frequency of support on a 4-point scale linked to standards: 0 (never), 1.33 ( rarely — a few times a 
year), 2.67 (sometimes — once or twice a month), and 4 (often — once or twice a week or almost daily). 

To examine the Content Focus of teachers’ activities, we asked each teacher who participated in 
technology-related professional development to indicate the degree of emphasis the activity placed on 
curriculum, instructional methods, and lesson development in their core-subject area. Teachers’ 
responses were coded on a 5-point scale with 0 = no emphasis, 2 - minor emphasis, and 4 -major 
emphasis. As a measure of professional development Coherence, each core teacher who attended 
technology-related events indicated the extent to which the activity was consistent with the their goals 
for professional development, was based explicitly on what the teacher had learned in earlier 
professional development experiences, was followed up with activities that built on what the teacher 
learned in the professional development activity, was aligned with state or district standards and 
curriculum frameworks and with state and distinct assessments. To measure this indicator, teachers 
used a 5-point scale ranging from 0 (not at all) to 4 (to a great extent). A Professional Development 
Index was generated by averaging z scores for each of the four professional development elements. 


121 



Extent of Implementation 

Classroom Immersion. The technology immersion packages included a variety of instructional and 
assessment resources designed to extend, supplement, or enhance core-subject teaching and learning. 
Wireless laptops, for example, were loaded with productivity software (i.e., either Appleworks or 
Microsoft Office) for students to use as a learning tool. Teachers and students also received a variety of 
digital resources and formative assessments to support content-area instruction and learning activities. 
Indicators for Classroom Immersion, accordingly, assessed the extent to which core-subject teachers at 
immersion campuses utilized resources and embraced practices consistent with the technology 
immersion model. Classroom Immersion is measured by five elements: Technology Integration, 
Learner-Centered Instruction, Student Classroom Activities, Communication, and Professional 
Productivity. Measures of Technology Integration (10 items) and Learner-Centered Instruction (4 
items) are scale scores adapted from the Levels of Technology Implementation (LoTi) Questionnaire. 
Core teachers indicated the extent to which statements related to Technology Integration (e.g., I alter 
my instructional practices to support higher order thinking through technology) and Learner-Centered 
Instruction (e.g., I have students use information and inquiry skills) are true on a 5-point scale, 
including 0 ( not true of me now), 1 to 3 ( somewhat true of me now), and 4 {very true of me now). 

Because teachers influence students’ classroom opportunities to use technology for learning academic 
content, we also used items from teacher surveys as a way to assess the extent to which teachers had 
students use various technology applications in core-subject classrooms (Student Classroom 
Activities). For example, survey items gauged how often students’ used a word processor to write a 
story or used software to learn and practice skills. Teachers’ responses were converted to a 5-point 
scale tied to immersion standards. Responses indicated how often students’ in a typical class used 
technology in particular ways: 0 (never), 1.33 ( rarely — a few times a year), 2.67 ( sometimes — once or 
twice a month), 4.00 ( often — once or twice a week — or almost daily). 

Teachers at immersion schools also are expected to use technology as a communication tool. 
Communication that advances student learning involves sending email to students, parents, or 
colleagues, or posting information and assignments on a class or school website. Technology also 
provides a way to improve teachers’ Professional Productivity, including the use of technology for 
puiposes such as keeping records, analyzing data, developing lessons, or delivering information. Scale 
scores for Communication (4 items) and Professional Productivity (11 items) arc comprised of teacher 
responses on a 5-point scale indicating the frequency of activities: 0 (never) to 4.00 (almost daily). The 
Classroom Immersion Index was generated by averaging z scores for each of the five elements 
described above. 

Student Access and Use. This indicator gauged the extent of student access to laptop computers as 
well as the frequency of students’ laptop use for learning in core-content classrooms and at home. 
Three elements — Laptop Access Days, Core-Content Learning, and Flome Learning — contribute to the 
component score. First, in an immersion school, students are expected to have access to wireless 
laptops for the entire school year. Our measure of Laptop Access was calculated as the number of days 
out of the 180-day school year that students actually had laptops available for use. Information for the 
indicator comes from an analysis of student survey items in which students indicated whether the 
school provided a laptop for student use, and if provided, how many days the laptop had been taken 
away (e.g., for misuse, misbehavior, failure to complete assignments, bad grades, or repairs). Student 
access scores, which could range from 0 days (no laptop) to 180 days (laptop available the full school 
year), were converted to the 0-4.00 continuous scale to measure progress toward the immersion 
standard. A Laptop Access Index was generated by transforming the continuous score to a z score. 
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The potential for laptops to affect achievement depends largely on students’ opportunities to use 
technology for learning core academic content. Consequently, we used items from student surveys (4) 
to assess the frequency with which students used technology resources in their English/language aits, 
mathematics, science, and social studies classrooms (Core-Content Learning). Students’ responses 
were converted to a 4-point frequency scale tied to standards: 0 ( never or rarely — a few times a year), 
1.33 ( sometimes — once or twice a month), 2.67 ( often — once or twice a week), and 4 ( almost daily). A 
Core-Content Learning Index was generated by transforming the scale score to a z score. 

Additionally, on a fully immersed campus, students should have access to their wireless laptops for 
learning both within and outside of school. Information for the measure of Home Learning comes 
from student survey items in which students indicated whether the school provided a laptop for student 
use, how often the student could take a laptop home, and if a laptop could be taken home, how often it 
was used for homework in core subjects or for learning games. A student’s use of the laptop for home 
learning was rated on a 6-point scale: 0 ( no access to laptop outside of school), 1 ( restricted or full 
access to laptop outside of school), plus up to 5 additional points if a student used their laptop for 
homework in ELA, math, science, or social studies, or for learning games. Students’ scores were 
converted to the 0-4.00 scale as a measure of progress toward immersion standards, and a z score was 
generated. We generated the Student Access and Use Index by averaging z scores for each of the three 
elements described above. 
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Table D.l. Scoring Rubrics for Measuring the Implementation Fidelity of Technology Immersion — Year 4 (Continued) 
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Table D.l. Scoring Rubrics for Measuring the Implementation Fidelity of Technology Immersion -Year 4 (Continued) 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators ( Continued ) 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators ( Continued ) 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators ( Continued ) 
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Appendix E 

Technical Appendix — Hierarchical Linear Modeling (HLM) 


Effects of Technology Immersion on Teachers and Teaching (Chapter 4) 

Researchers estimated the effects of immersion on teacher mediating variables using three-level 
hierarchical linear growth models. In our models, we posit that school poverty is related to teachers’ 
initial status and yearly growth rate. Statistical details are provided in Tables E.l, E.2, and E.3. The 
models’ simplicity aids in the interpretation of effects. More complex models, controlling for teacher 
demographic characteristics (gender, ethnicity, experience), described subsequently in Tables E.4, E.5 
and E.6, estimated nearly identical immersion growth coefficients. 


Table E.l. Descriptive Statistics for Teacher Variables: HLM Models with School Poverty 


1 Variable Name 

N 

Mean 

si) ; 

Repeated Measures Descriptive Statistics (Level 1) 

Survey Time 

10,685 

2.00 

1.41 

Technology Proficiency 

5,541 

4.96 

1.40 

Professional Productivity 

5,484 

3.30 

0.73 

Technology Integration 

5,219 

3.79 

1.58 

Learner-Centered Instruction 

5,390 

4.19 

1.38 

Resistance to Integration 

5,426 

2.36 

1.40 

Student Classroom Activities 

5,448 

2.19 

0.81 

Collaboration 

5,487 

2.53 

0.78 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School percent economically disadvantaged 

42 

68.52 

16.83 


Table E.2. Immersion (Fixed) Effect Analyses of Teacher Mediating Variables: 
HLM Models with School Poverty 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

4.692 

0.081 

58.26*** 


Immersion dummy 

-0.165 

0.112 

-1.48 


School poverty 

0.001 

0.003 

0.16 


Growth rate 

0.147 

0.014 

10.53*** 


Immersion dummy 

0.138 

0.023 

6.06*** 


School poverty 

-0.002 

0.001 

-3.23** 

Professional Productivity 


Initial status (fall 2004) 

3.011 

0.058 

51.53*** 


Immersion dummy 

-0.062 

0.078 

-0.80 


School poverty 

0.001 

0.002 

0.57 


Growth rate 

0.110 

0.009 

17*** 


Immersion dummy 

0.069 

0.014 

4 92*** 


School poverty 

0.000 

0.000 

-0.66 

Technology Integration 


Initial status (fall 2004) 

2.847 

0.073 

38.95*** 


Immersion dummy 3 

0.445 

0.097 

4 ^ i*** 


School poverty 

0.010 

0.003 

3.50** 


Growth rate 3 

0.002 

0.532 

0.00 


Immersion dummy 3 

0.105 

0.093 

1.14 


School poverty 3 

-0.005 

0.002 

-2.30* 


Initial status 3 

0.103 

0.187 

0.55 


Continued 
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Table E.2. Immersion (Fixed) Effect Analysis of Teacher Mediating Variables (Continued) 


1 Learner-Centered Instruction | 


Initial status (fall 2004) 

3.683 

0.065 

56.37*** 


Immersion dummy 

0.036 

0.092 

0.39 


School poverty 

0.006 

0.002 

2.62* 


Growth rate 

0.199 

0.017 

11.75*** 


Immersion dummy 

0.110 

0.032 

3.42** 


School poverty 

- 0.002 

0.001 

-2.08* 

Resistance to Integration 


Initial status (fall 2004) 

2.463 

0.055 

44.57*** 


Immersion dummy b 

-0.295 

0.070 

_4 24*** 


School poverty 

-0.003 

0.002 

-i.75 f 


Growth rate 

0.011 

0.013 

0.86 


Immersion dummy b 

0.024 

0.023 

1.03 


School poverty 

0.002 

0.000 

3.69** 

Student Classroom Activities 


Initial status (fall 2004) 

1.858 

0.047 

39.52*** 


Immersion dummy 0 

0.161 

0.061 

2.65* 


School poverty 

0.004 

0.002 

2.56* 


Growth rate 

0.073 

0.012 

5.90*** 


Immersion dummy 0 

0.101 

0.018 

5.60*** 


School poverty 

- 0.001 

0.001 

-2.51* 

Collaboration 


Initial status (fall 2004) 

2.292 

0.048 

48.04*** 


Immersion dummy d 

0.143 

0.062 

2.29* 


School poverty 

0.004 

0.002 

2.30* 


Growth rate 

0.055 

0.014 

2 97 *** 


Immersion dummy d 

0.018 

0.019 

0.95 


School poverty 

0.000 

0.001 

0.15 


t p < .10; *p < .05; **p < .01; ***/? < .001. 

a Immersion teachers had significantly higher initial technology integration scores. A latent variable regression was run to control for 
the effect of this initial difference on the growth rate. The latent variable regression indicated that the immersion effect became not 
significant after controlling for initial differences. Thus, the coefficients from the latent variable regression model are reported here. 
b Immersion teachers had significantly lower initial resistance to integration scores. A latent variable regression was run to control for 
the effect of this initial difference on the growth rate. The immersion effect was not a significant predictor of the growth rate with 
and without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
c Immersion teachers had significantly higher initial student classroom activities scores. A latent variable regression was run to control 
for the effect of this initial difference on the growth rate. The immersion effect was a significant predictor of the growth rate with 
and without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
d Immersion teachers had significantly higher initial teacher collaboration scores. A latent variable regression was run to control for 
the effect of this initial difference on the growth rate. The immersion effect was not a significant predictor of the growth rate with 
and without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
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Table E.3. Variance Decomposition from Conditional HLM Growth Models of Teacher 
Mediating Variables (with School Poverty) 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.3272 




Level-2 individual initial status 

1.7749 

1377 

9689.69 

0.000 

Level-2 individual growth rate 

0.0299 

1377 

2066.54 

0.000 

Level-2 school initial status 

0.0577 

39 

94.09 

0.000 

Level-2 school growth rate 

0.0018 

39 

71.96 

0.001 

Professional Productivity 

Level- 1 temporal variation 

0.1580 




Level-2 individual initial status 

0.3185 

1368 

4356.51 

0.000 

Level-2 individual growth rate 

0.0094 

1368 

1898.50 

0.000 

Level-2 school initial status 

0.0447 

39 

163.93 

0.000 

Level-2 school growth rate 

0.0006 

39 

69.27 

0.002 

Technology Integration 

Level- 1 temporal variation 

0.7740 




Level-2 individual initial status 

1.2219 

1326 

3609.38 

0.000 

Level-2 individual growth rate 

0.0299 

1326 

1761.13 

0.000 

Level-2 school initial status 

0.0341 

39 

70.04 

0.002 

Level-2 school growth rate 

0.0077 

39 

97.43 

0.000 

Learner-Centered Instruction 

Level- 1 temporal variation 

0.7208 




Level-2 individual initial status 

0.9869 

1355 

3392.48 

0.000 

Level-2 individual growth rate 

0.0224 

1355 

1635.95 

0.000 

Level-2 school initial status 

0.0330 

39 

77.57 

0.000 

Level-2 school growth rate 

0.0043 

39 

86.90 

0.000 

Resistance to Integration 

Level- 1 temporal variation 

0.9160 




Level-2 individual initial status 

0.7162 

1362 

2592.57 

0.000 

Level-2 individual growth rate 

0.0320 

1362 

1727.09 

0.000 

Level-2 school initial status 

0.0058 

39 

49.46 

0.122 

Level-2 school growth rate 

0.0002 

39 

44.90 

0.238 

Student Classroom Activities 

Level- 1 temporal variation 

0.2316 




Level-2 individual initial status 

0.2861 

1364 

3195.86 

0.000 

Level-2 individual growth rate 

0.0047 

1364 

1725.86 

0.000 

Level-2 school initial status 

0.0218 

39 

110.90 

0.000 

Level-2 school growth rate 

0.0015 

39 

89.94 

0.000 

Collaboration 

Level- 1 temporal variation 

0.2554 




Level-2 individual initial status 

0.2713 

1372 

2932.92 

0.000 

Level-2 individual growth rate 

0.0102 

1372 

1742.01 

0.000 

Level-2 school initial status 

0.0233 

39 

111.78 

0.000 

Level-2 school growth rate 

0.0015 

39 

83.64 

0.000 
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Researchers also used HLM growth models to estimate immersion effects on teacher mediating variables, 
controlling for teacher characteristics. Statistical details for these models are provided in Tables E.4, E.5 
and E.6. 

Table E.4. Descriptive Statistics for Teacher Variables: 

HLM models with Teacher Characteristics 


1 Variable Name 

N 

Mean 

SD 1 

Repeated Measures Descriptive Statistics (Level 1) 

Time 

9,430 

2.00 

1.41 

Technology Proficiency 

5,403 

4.95 

1.40 

Professional Productivity 

5,351 

3.29 

0.73 

Technology Integration 

5,088 

3.78 

1.58 

Learner-Centered Instruction 

5,256 

4.18 

1.37 

Resistance to Integration 

5,289 

2.35 

1.39 

Student Classroom Activities 

5,316 

2.18 

0.80 

Collaboration 

5,353 

2.53 

0.78 

Teacher-Level Descriptive Statistics (Level 2) 

Male 

1,886 

0.33 

0.47 

Hispanic 

1,886 

0.35 

0.48 

African American 

1,886 

0.05 

0.22 

Experience 

1,886 

10.66 

9.61 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 


Table E.5. Immersion (Fixed) Effect Analyses of Teacher-Level Variables: 
HLM Models with Teacher Characteristics 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

4.845 

0.073 

66.34*** 


Immersion dummy 3 

-0.186 

0.083 

-2.23* 


Male 

-0.090 

0.087 

-1.04 


Hispanic 

-0.166 

0.074 

-2.24* 


African American 

-0.017 

0.135 

-0.13 


Experience 

-0.055 

0.004 

_12 87 *** 


Growth rate 

0.123 

0.015 

8 og*** 


Immersion dummy 3 

0.139 

0.026 

5.32*** 


Male 

-0.018 

0.018 

- 1.01 


Hispanic 

0.014 

0.022 

0.63 


African American 

0.008 

0.023 

0.36 


Experience 

0.004 

0.001 

5.86*** 

Professional Productivity 


Initial status (fall 2004) 

3.093 

0.058 

52.90*** 


Immersion dummy 

-0.061 

0.073 

-0.85 


Male 

- 0.200 

0.051 

-3 92*** 


Hispanic 

0.032 

0.046 

0.70 


African American 

0.095 

0.076 

1.25 


Experience 

-0.014 

0.003 

-5 12*** 


Growth rate 

0.093 

0.011 

8.30*** 


Immersion dummy 

0.067 

0.014 

4 ^ 9 *** 


Male 

0.014 

0.014 

1.05 


Hispanic 

0.011 

0.010 

1.11 


African American 

0.018 

0.022 

0.81 


Experience 

0.001 

0.001 

1.74 t 


(Continued) 
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Table E.5. Immersion (Fixed) Effect Analysis of Teacher-Level Variables (Continued) 


1 Technology Integration | 


Initial status (fall 2004) 

2.855 

0.076 

37.76*** 


Immersion dummy b 

0.441 

0.099 

4 44*** 


Male 

-0.187 

0.068 

-2.74** 


Hispanic 

0.301 

0.086 

3.50** 


African American 

0.513 

0.091 

5.62*** 


Experience 

-0.017 

0.004 

_4 24*** 


Growth rate 

0.296 

0.029 

10 14*** 


Immersion dummy b 

0.145 

0.042 

3.49** 


Male 

-0.004 

0.022 

-0.17 


Hispanic 

-0.055 

0.024 

-2.27* 


African American 

-0.008 

0.040 

-0.21 


Experience 

0.003 

0.001 

2.55* 

Learner-Centered Instruction 


Initial status (fall 2004) 

3.757 

0.064 

58.35*** 


Immersion dummy 

0.021 

0.087 

0.24 


Male 

-0.240 

0.069 

-3.47** 


Hispanic 

0.166 

0.065 

2.56* 


African American 

0.484 

0.103 

4.68*** 


Experience 

-0.025 

0.004 

-6.61*** 


Growth rate 

0.175 

0.019 

9 30*** 


Immersion dummy 

0.104 

0.034 

3.03** 


Male 

0.003 

0.027 

0.12 


Hispanic 

0.012 

0.023 

0.53 


African American 

0.022 

0.037 

0.60 


Experience 

0.003 

0.001 

2.58* 

Resistance to Integration 


Initial status (fall 2004) 

2.416 

0.061 

39.51*** 


Immersion dummy 0 

-0.319 

0.061 

-5.28*** 


Male 

0.410 

0.076 

5 42*** 


Hispanic 

-0.325 

0.060 

-5.40*** 


African American 

-0.342 

0.110 

-3.10** 


Experience 

0.013 

0.004 

3.20** 


Growth rate 

-0.003 

0.019 

-0.17 


Immersion dummy 0 

0.024 

0.023 

1.05 


Male 

0.033 

0.026 

1.25 


Hispanic 

0.060 

0.019 

3.10** 


African American 

0.038 

0.058 

0.65 


Experience 

-0.004 

0.001 

_2 7i** 

Student Classroom Activities 


Initial status (fall 2004) 

1.817 

0.047 

38.78*** 


Immersion dummy d 

0.160 

0.056 

2.85** 


Male 

-0.045 

0.052 

-0.85 


Hispanic 

0.185 

0.041 

4 49*** 


African American 

0.299 

0.077 

3 9i*** 


Experience 

-0.004 

0.002 

-2.11* 


Growth rate 

0.068 

0.014 

4 27*** 


Immersion dummy d 

0.098 

0.020 

4 94*** 


Male 

0.017 

0.017 

1.04 


Hispanic 

-0.016 

0.013 

-1.22 


African American 

-0.002 

0.025 

-0.08 


Experience 

0.000 

0.000 

-0.27 


(Continued) 
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Table E.5. Immersion (Fixed) Effect Analysis of Teacher-Level Variables (Continued) 


1 Collaboration | 

Initial status (fall 2004) 

2.265 

0.043 

52 91*** 

Immersion dummy 0 

0.145 

0.056 

2.58* 

Male 

-0.022 

0.042 

-0.54 

Hispanic 

0.145 

0.047 

3.08** 

African American 

0.332 

0.072 

4 g 4*** 

Experience 

-0.007 

0.002 

-3.58** 

Growth rate 

0.047 

0.015 

3.06** 

Immersion dummy 0 

0.014 

0.018 

0.79 

Male 

0.010 

0.011 

0.90 

Hispanic 

0.004 

0.015 

0.29 

African American 

-0.029 

0.027 

-1.08 

Experience 

0.000 

0.001 

0.59 


f p < .10; *p < .05; **p < .01; ***p < .001. 

a Immersed teachers had significantly lower initial technology proficiency scores. A latent variable regression was run to control for 
the effect of this initial difference on the growth rate. The immersion effect was a significant predictor of the growth rate with and 
without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
b Immersed teachers had significantly higher technology integration scores. A latent variable regression was run to control for the 
effect of this initial difference on the growth rate. The immersion effect was a significant predictor of the growth rate with and 
without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
c Immersed teachers had significantly lower initial resistance to integration scores. A latent variable regression was run to control for 
the effect of this initial difference on the growth rate. The immersion effect was not a significant predictor of the growth rate with 
and without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
d Immersion teachers had significantly higher initial student classroom activities scores. A latent variable regression was run to control 
for the effect of this initial difference on the growth rate. The immersion effect was a significant predictor of the growth rate with 
and without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
e Immersion teachers had significantly higher initial teacher collaboration scores. A latent variable regression was run to control for 
the effect of this initial difference on the growth rate. The immersion effect was not a significant predictor of the growth rate with 
and without controlling for initial differences. Thus, the coefficients from the original growth model are reported here. 
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Table E.6. Variance Decomposition from Conditional HLM Growth Models of Teacher 
Mediating Variables (with Teacher Characteristics) 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.3282 




Level-2 individual initial status 

1.5267 

1357 

8450.52 

0.000 

Level-2 individual growth rate 

0.0280 

1357 

2014.06 

0.000 

Level-2 school initial status 

0.0186 

40 

64.26 

0.009 

Level-2 school growth rate 

0.0032 

40 

101.56 

0.000 

Professional Productivity 

Level- 1 temporal variation 

0.1577 




Level-2 individual initial status 

0.2915 

1350 

4076.97 

0.000 

Level-2 individual growth rate 

0.0089 

1350 

1864.75 

0.000 

Level-2 school initial status 

0.0376 

40 

149.66 

0.000 

Level-2 school growth rate 

0.0007 

40 

70.95 

0.002 

Technology Integration 

Level- 1 temporal variation 

0.7729 




Level-2 individual initial status 

1.1650 

1307 

3461.82 

0.000 

Level-2 individual growth rate 

0.0295 

1307 

1739.92 

0.000 

Level-2 school initial status 

0.0373 

40 

70.72 

0.002 

Level-2 school growth rate 

0.0107 

40 

119.00 

0.000 

Learner-Centered Instruction 

Level- 1 temporal variation 

0.7187 




Level-2 individual initial status 

0.9173 

1336 

3190.64 

0.000 

Level-2 individual growth rate 

0.0210 

1336 

1602.35 

0.000 

Level-2 school initial status 

0.0263 

40 

74.50 

0.001 

Level-2 school growth rate 

0.0059 

40 

104.31 

0.000 

Resistance to Integration 

Level- 1 temporal variation 

0.9170 




Level-2 individual initial status 

0.6278 

1342 

2453.70 

0.000 

Level-2 individual growth rate 

0.0305 

1342 

1702.84 

0.000 

Level-2 school initial status 

0.0024 

40 

42.51 

0.363 

Level-2 school growth rate 

0.0003 

40 

41.46 

0.407 

Student Classroom Activities 

Level- 1 temporal variation 

0.2300 




Level-2 individual initial status 

0.2771 

1345 

3118.26 

0.000 

Level-2 individual growth rate 

0.0047 

1345 

1706.70 

0.000 

Level-2 school initial status 

0.0157 

40 

95.91 

0.000 

Level-2 school growth rate 

0.0022 

40 

107.71 

0.000 

Collaboration 

Level- 1 temporal variation 

0.2546 




Level-2 individual initial status 

0.2644 

1353 

2867.76 

0.000 

Level-2 individual growth rate 

0.0096 

1353 

1718.12 

0.000 

Level-2 school initial status 

0.0154 

40 

93.97 

0.000 

Level-2 school growth rate 

0.0015 

40 

83.79 

0.000 
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Effects of Technology Immersion on Students and Learning (Chapter 5) 

For the results reported in Chapter 5, researchers analyzed the effects of immersion on student mediating 
variables for Cohorts 2 and 3 using three-level FILM models. 

Effects on Mediating Variables 

In spring 2007, student surveys were not administered at two treatment campuses and one control 
campus. We used AMOS 7.0 to perform model-based imputations to predict these missing scores. 
Specifically, for Cohort 2, we imputed scores for the three school technology scales and for the three 
self-perception scales. For Cohort 3, we also imputed scores for the three school technology scales and for 
only two self-perception scales because self-directed learning was not studied in Cohort 3. Our 
student-level model predicted the spring 2007 student scale score from the spring 2006 scale score for 
Cohort 2 and from the fall 2006 scale score for Cohort 3, gender (1 if female, 0 if male), 

African-American status (1 if African American, 0 if not), Hispanic status (1 if Hispanic, 0 if not), 
economic status (1 if on free- or reduced-lunch, 0 if not), and immersion status (1 if the student attended 
an immersion campus, 0 if he or she attended a control campus). The result was five complete datasets for 
each scale for Cohorts 2 and 3. 

These multiply-imputed datasets were then analyzed using HLM 6.04. (Note that HLM results from 10 
imputed datasets were compared to the results from 5 imputed datasets, and there were essentially no 
differences in the coefficients. The reduced number of imputed datasets made the HLM analyses 
mechanically easier to run.) Specifically, researchers used three-level HLM growth models, with controls 
for school poverty (percentage of economically disadvantaged students) and student poverty 
(qualification for free- or reduced-price lunch). The models’ simplicity aids in the interpretation of 
effects. Statistical details are provided in Tables E.7, E.8, and E.9 for analyses of mediating variables for 
Cohort 2. 


Table E.7. Descriptive Statistics for Student Variables, Cohort 2 


Variable Name 

N 

Mean a 

SD 

Repeated Measure Descriptive Statistics (Level 1) 

Time 

18,108 

1.50 

1.12 

Time (SLI) 

17,220 

1.50 

1.12 

Technology Proficiency score 

14,720 

3.28 

0.91 

Classroom Activities score 

14,160 

2.37 

0.84 

Technical Problems score 

14,311 

2.41 

0.93 to 0.94 

Small-Group Work score 

14,151 

2.80 

0.88 

School Satisfaction score 

14,305 

3.69 

0.76 

Self-Directed Learning score 

13,276 

4.49 

0.74 

Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,527 

0.67 

0.47 

Eco. disadvantaged (SLI) 

4,305 

0.66 

0.47 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 


J Range of imputed means is listed when means differed across imputations. 
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Table E.8. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 2 



School-Level 

Gamma 

Standard 


School-Level Scale 

Analysis 

Coefficient 

Error 

t 

Technology Proficiency 


Initial status (fall 2005) 

2.982 

0.055 

54.45*** 


Immersion dummy 

- 0.002 

0.076 

- 0.02 


School poverty 

0.002 

0.002 

0.93 


Disadvantaged 

-0.264 

0.045 

-5.90*** 


Growth rate 

0.256 

0.023 

11 34 *** 


Immersion dummy 

0.094 

0.031 

3.06** 


School poverty 

- 0.001 

0.001 

-0.59 


Disadvantaged 

0.004 

0.018 

0.25 

Self-Directed Learning 


Initial status (fall 2005) 

4.721 

0.044 

107.37*** 


Immersion dummy 

-0.043 

0.055 

-0.78 


School poverty 

0.003 

0.002 

1.58 


Disadvantaged 

-0.096 

0.026 

-3.64** 


Growth rate 

-0.142 

0.013 

-11.05*** 


Immersion dummy 

0.013 

0.018 

0.74 


School poverty 

0.000 

0.001 

0.30 


Disadvantaged 

0.002 

0.011 

0.21 

School Satisfaction 


Initial status (fall 2005) 

3.829 

0.029 

130.76*** 


Immersion dummy 

0.038 

0.037 

1.03 


School poverty 

0.000 

0.001 

-0.36 


Disadvantaged 

-0.140 

0.022 

-6.31*** 


Growth rate 

-0.058 

0.012 

-4.90*** 


Immersion dummy 

-0.018 

0.018 

-0.99 


School poverty 

0.001 

0.000 

2.56* 


Disadvantaged 

0.029 

0.011 

2.71** 

Classroom Activities (with technology) 


Initial status (fall 2005) 

2.058 

0.052 

39.61*** 


Immersion dummy 3 

0.247 

0.082 

3.00** 


School poverty 

0.006 

0.002 

2.64* 


Disadvantaged 

-0.005 

0.039 

- 0.12 


Growth rate 

0.099 

0.029 

3.41** 


Immersion dummy 3 

0.092 

0.041 

2.26* 


School poverty 

-0.003 

0.001 

-2.35* 


Disadvantaged 

0.029 

0.015 

oo 

00 

-+ 

Small-Group Work 


Initial status (fall 2005) 

2.762 

0.050 

55.26*** 


Immersion dummy 

-0.024 

0.070 

-0.34 


School poverty 

0.002 

0.002 

1.28 


Disadvantaged 

0.002 

0.045 

0.04 


Growth rate 

0.007 

0.017 

0.43 


Immersion dummy 

0.091 

0.029 

3.12** 


School poverty 

- 0.001 

0.001 

-1.15 


Disadvantaged 

0.003 

0.017 

0.20 


( Continued ) 
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Table E.8. Immersion (Fixed) Effect Analysis of Student Variables, Cohort 2 (Continued) 


Technical Problems 

Initial status (fall 2005) 

2.209 

0.044 

50.50*** 

Immersion dummy b 

-0.284 

0.066 

-4 32*** 

School poverty 

0.005 

0.002 

2.31* 

Disadvantaged 

-0.058 

0.028 

-2.07* 

Growth rate 

0.145 

0.028 

5.15*** 

Immersion dummy b 

0.198 

0.036 

5.57*** 

School poverty 

-0.002 

0.001 

-2.14* 

Disadvantaged 

0.019 

0.015 

1.33 


+ p < .10; *p < .05; **p < .01; ***p < .001. 


‘‘Immersion students had significantly higher initial classroom activities scores. A latent variable regression was run to control 
for the effect of this initial difference on the growth rate. The immersion effect was a significant predictor of the growth rate 
with and without controlling for initial differences. Thus, the coefficients from original growth model are reported here. 
b Immersion students had significantly lower initial technical problems scores. A latent variable regression was run to control 
for the effect of this initial difference on the growth rate. The immersion effect was a significant predictor of the growth rate 
with and without controlling for initial differences. Thus, the coefficients from original growth model are reported here. 


Table E.9. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 2 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.3147 



Level-2 individual initial status 

0.4328 

4154 

11431.32 

0.000 

Level-2 individual growth rate 

0.0294 

4154 

5858.72 

0.000 

Level-3 school initial status 

0.0511 

39 

305.41 

0.000 

Level-3 school growth rate 

0.0084 

39 

310.70 

0.000 

Self-Directed Learning 

Level- 1 temporal variation 

0.2250 



Level-2 individual initial status 

0.3096 

3684 

10269.22 

0.000 

Level-2 individual growth rate 

0.0232 

3684 

5187.67 

0.000 

Level-3 school initial status 

0.0233 

39 

214.15 

0.000 

Level-3 school growth rate 

0.0019 

39 

125.16 

0.000 

School Satisfaction 

Level- 1 temporal variation 

0.3591 



Level-2 individual initial status 

0.2084 

4101 

7025.10 

0.000 

Level-2 individual growth rate 

0.0175 

4101 

4956.99 

0.000 

Level-3 school initial status 

0.0074 

39 

98.11 

0.000 

Level-3 school growth rate 

0.0017 

39 

97.32 

0.000 

Classroom Activities 

Level- 1 temporal variation 

0.4495 



Level-2 individual initial status 

0.1890 

4069 

6093.71 

0.000 

Level-2 individual growth rate 

0.0141 

4069 

4671.39 

0.000 

Level-3 school initial status 

0.0615 

39 

378.97 

0.000 

Level-3 school growth rate 

0.0159 

39 

493.31 

0.000 


( Continued ) 
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Table E.9. Variance Decomposition from Conditional HLM Models of Student Mediating 
Variables, Cohort 2 (Continued) 


Small-Group Work ! 

Level- 1 temporal variation 

0.5443 




Level-2 individual initial status 

0.2461 

4075 

6194.30 

0.000 

Level-2 individual growth rate 

0.0286 

4075 

4883.41 

0.000 

Level-3 school initial status 

0.0398 

39 

230.69 

0.000 

Level-3 school growth rate 

0.0063 

39 

184.97 

0.000 

Technical Problems 

Level- 1 temporal variation 

0.5597 




Level-2 individual initial status 

0.1590 

4100 

5565.16 

0.000 

Level-2 individual growth rate 

0.0236 

4100 

4966.56 

0.000 

Level-3 school initial status 

0.0336 

39 

211.20 

0.000 

Level-3 school growth rate 

0.0108 

39 

241.79 

0.000 


Statistical details are provided in Tables E.10, E. 11, and E.12 for analyses of mediating variables for 
Cohort 3. 


Table E.10. Descriptive Statistics for Student Variables, Cohort 3 


Variable Name 

N 

Mean a 

SD 

Repeated Measure Descriptive Statistics (Level 1) 

Time 

13,158 

LOO 

0.82 

Technology Proficiency score 

11,354 

4.30 

0.90 

Classroom Activities score 

11,178 

2.34 

0.86 

Technical Problems score 

11,170 

2.28 

0.92 to 0.93 

Small-Group Work score 

11,109 

2.80 

0.88 to 0.89 

School Satisfaction score 

11,115 

3.71 

0.75 

Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,386 

0.69 

0.46 

Eco. disadvantaged (SLI) 




School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 


J Range of imputed means is listed when means differed across imputations. 
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Table E.11. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 3 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status ( fall 2006) 

2.951 

0.057 

52.10*** 


Immersion dummy 

-0.019 

0.084 

- 0.22 


School poverty 

0.001 

0.002 

0.45 


Disadvantaged 

-0.145 

0.043 

-3.38** 


Growth rate 

0.266 

0.019 

13.75*** 


Immersion dummy 

0.168 

0.036 

4.68*** 


School poverty 

- 0.001 

0.001 

-1.37 


Disadvantaged 

- 0.010 

0.018 

-0.57 

School Satisfaction 


Initial status ( fall 2006) 

3.797 

0.027 

138.60*** 


Immersion dummy 

0.050 

0.038 

1.33 


School poverty 

- 0.001 

0.001 

-1.05 


Disadvantaged 

-0.084 

0.022 

-3.81*** 


Growth rate 

-0.092 

0.022 

-4 29*** 


Immersion dummy 

0.022 

0.031 

0.69 


School poverty 

0.001 

0.001 

0.95 


Disadvantaged 

0.035 

0.020 

1.74 

Classroom Activities (with technology) 


Initial status ( fall 2006) 

1.967 

0.088 

22.45*** 


Immersion dummy 2 

0.436 

0.119 

3.67** 


School poverty 

0.004 

0.004 

1.13 


Disadvantaged 

0.002 

0.063 

0.03 


Growth rate 2 

1.067 

0.126 

g 47 *** 


Immersion dummy 2 

0.217 

0.052 

4 20 *** 


School poverty 2 

-0.003 

0.001 

0.044* 


Disadvantaged 

0.059 

0.037 

1.58 


Initial status 2 

-0.464 

0.062 

_7 49 *** 

Small-Group Work 


Initial status ( fall 2006) 

2.785 

0.067 

41 57*** 


Immersion dummy 

-0.015 

0.086 

-0.18 


School poverty 

0.002 

0.003 

0.79 


Disadvantaged 

-0.031 

0.053 

-0.58 


Growth rate 

-0.039 

0.034 

-1.17 


Immersion dummy 

0.144 

0.047 

3.09** 


School poverty 

- 0.001 

0.001 

-0.94 


Disadvantaged 

0.036 

0.029 

1.25 


( Continued ) 
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Table E.11. Immersion (Fixed) Effect Analysis of Student Variables, Cohort 3 
(Continued) 


Technical Problems 

Initial status (fall 2006) 

2.136 

0.057 

37.66*** 

Immersion dummy b 

-0.139 

0.072 

-1.92 f 

School poverty 

-0.003 

0.002 

-1.67 

Disadvantaged 

-0.047 

0.043 

-1.11 

Growth rate 

0.152 

0.037 

4 12*** 

Immersion dummy b 

0.235 

0.048 

4 91*** 

School poverty 

0.001 

0.002 

0.39 

Disadvantaged 

-0.001 

0.019 

-0.05 


< .10; *p < .05; **p < .01; ***p < .001. 

‘‘Immersion students had significantly higher initial classroom activities scores. A latent variable regression was 
run to control for the effect of this initial difference on the growth rate. The latent variable regression indicated 
that the immersion effect became significant after controlling for initial differences. Thus, the coefficients from the 
latent variable regression model are reported here. 

b Immersion students had significantly lower initial technical problems scores. A latent variable regression was run 
to control for the effect of this initial difference on the growth rate. The immersion effect was a significant 
predictor of the growth rate with and without controlling for initial differences. Thus, the coefficients from original 
growth model are reported here. 


Table E.12. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 3 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.2964 




Level-2 individual initial status 

0.4858 

3996 

11277.03 

0.000 

Level-2 individual growth rate 

0.0495 

3996 

5243.04 

0.000 

Level-3 school initial status 

0.0548 

39 

313.12 

0.000 

Level-3 school growth rate 

0.0084 

39 

176.43 

0.000 

School Satisfaction 

Level- 1 temporal variation 

0.3465 




Level-2 individual initial status 

0.1805 

3922 

6093.07 

0.000 

Level-2 individual growth rate 

0.0249 

3922 

4426.58 

0.000 

Level-3 school initial status 

0.0076 

39 

102.19 

0.000 

Level-3 school growth rate 

0.0057 

39 

135.20 

0.000 

Classroom Activities 

Level- 1 temporal variation 

0.43751 




Level-2 individual initial status 

0.2095 

3947 

5969.44 

0.000 

Level-2 individual growth rate 

0.0241 

3947 

4360.33 

0.000 

Level-3 school initial status 

0.1348 

39 

885.50 

0.000 

Level-3 school growth rate 

0.0473 

39 

609.68 

0.000 
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Table E.12. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 3 (Continued) 


Small-Group Work 

Level- 1 temporal variation 

0.5243 




Level-2 individual initial status 

0.2422 

3927 

5857.82 

0.000 

Level-2 individual growth rate 

0.0302 

3927 

4341.39 

0.000 

Level-3 school initial status 

0.0663 

39 

445.03 

0.000 

Level-3 school growth rate 

0.0160 

39 

222.70 

0.000 

Technical Problems 

Level- 1 temporal variation 

0.5548 




Level-2 individual initial status 

0.1473 

3942 

5131.19 

0.000 

Level-2 individual growth rate 

0.0277 

3942 

4425.55 

0.000 

Level-3 school initial status 

0.0415 

39 

292.47 

0.000 

Level-3 school growth rate 

0.0171 

39 

234.29 

0.000 


Effects on School Attendance 

Comparable to analyses for student-level variables, we used three-level HLM growth models to estimate 
the effects of immersion on student attendance. Statistical details are provided in Tables E.13, E.14, and 
E.15. 


Table E.13. Descriptive Statistics for Student Attendance 


I Variable Name 

N 

Mean 

SD 1 

Cohort 2 Repeated Measures Descriptive Statistics (Level 1) 

Year 

16,816 

1.50 

1.12 

Attendance 

16,703 

96.69 

3.84 

Cohort 3 Repeated Measures Descriptive Statistics (Level 1) 

Year 

13,851 

1.00 

0.82 

Attendance 

13,742 

96.67 

3.80 

Cohort 2 Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,204 

0.73 

0.45 

Cohort 3 Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,617 

0.74 

0.44 

Cohorts 2 and 3 School-Level Descriptive 1 

Statistics 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

Percentage school poverty ( 2007-08) 

42 

68.52 

16.83 
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Table E.14. Immersion (Fixed) Effect Analyses of Student Attendance 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Cohort 2 






Initial attendance (2005) 

97.587 

0.144 

677.01*** 


Immersion dummy 

-0.232 

0.173 

-1.34 


School poverty 

0.014 

0.005 

2.59* 


Eco. disadvantaged 

-0.356 

0.124 

-2.86** 


Growth rate 

-0.189 

0.060 

-3.15*** 


Immersion dummy 

-0.059 

0.069 

-0.86 


School poverty 

0.001 

0.002 

0.40 


Eco. disadvantaged 

-0.288 

0.035 

-8.35*** 

Cohort 3 






Initial attendance (2006) 

97.600 

0.137 

711.39*** 


Immersion dummy 3 

-0.476 

0.237 

-2.01* 


School poverty 

0.022 

0.005 

4 12*** 


Eco. disadvantaged 

-0.567 

0.132 

_4 


Growth rate 

-0.295 

0.094 

-3.15** 


Immersion dummy 3 

-0.100 

0.118 

-0.85 


School poverty 

-0.001 

0.004 

-0.32 


Eco. disadvantaged 

-0.115 

0.048 

-2.40* 


'p < .10; *p < .05; **p < .01; ***p < .001. 

3 Immersed students had significantly lower initial 2006 attendance rates. A latent variable regression was run to 
control for the effect of this initial difference on the growth rate. The immersion effect was not a significant 
predictor of the growth rate with and without controlling for initial differences. Thus, the coefficients from the 
original growth model are reported here. 


Table E.15. Variance Decomposition from Conditional HLM Models of Student Attendance, 
Cohorts 2 and 3 


Cohort/ 
Random Effect 

Variance 

Component 

df 

X 2 

Pt 

Cohort 2 

Level- 1 temporal variation 

4.5174 




Level-2 individual initial status 

5.4200 

4161 

11188.54 

0.000 

Level-2 individual growth rate 

0.9423 

4161 

8450.03 

0.000 

Level-3 school initial status 

0.1469 

39 

102.02 

0.000 

Level-3 school growth rate 

0.0234 

39 

88.90 

0.000 

Cohort 3 

Level- 1 temporal variation 

4.9719 




Level-2 individual initial status 

5.6117 

4574 

10711.18 

0.000 

Level-2 individual growth rate 

0.7939 

4574 

6060.58 

0.000 

Level-3 school initial status 

0.4016 

39 

191.65 

0.000 

Level-3 school growth rate 

0.0870 

39 

148.07 

0.000 
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Effects of Technology Immersion on Student Achievement (Chapter 6) 


Researchers used three-level HLM growth models to estimate the effects of immersion on student 
academic achievement. Statistical details are provided for Cohort 2 students (eighth graders) in Tables 
E.16 through E.19, for Cohort 3 (seventh graders) in Tables E.20 through E.23. and for Cohort 1 students 
(ninth graders, post immersion) in Tables E.24 through E.26. 

Cohort 2 (Eighth Graders) 

Table E.16. Descriptive Statistics for TAKS Reading and Mathematics, 

Cohort 2 


1 Variable Name 

N 

Mean 

SD 1 

Repeated Measures Descriptive Statistics: Reading (Level 1) 

Time 

13,072 

1.50 

1.12 

TAKS Reading T score 

12,771 

48.90 

9.57 

Repeated Measures Descriptive Statistics: Mathematics (Level 1) 

Time 

13,072 

1.50 

1.12 

TAKS Mathematics T score 

12,745 

48.86 

9.28 

Student-Level Descriptive Statistics: Reading (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

3,268 

0.70 

0.46 

Student-Level Descriptive Statistics 

: Mathematics (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,268 

0.70 

0.46 

School-Level Descriptive Statistics: (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 


Table E.17. Descriptive Statistics for TAKS Science and Social Studies, 
Cohort 2 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

: Science (Level 1) 


Female 

3,268 

0.52 

0.50 

African American 

3,268 

0.07 

0.25 

Hispanic 

3,268 

0.69 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

3,268 

0.70 

0.46 

TAKS Science T score (2005) 

3,025 

49.09 

9.58 

TAKS Science T score (2008) 

3,211 

48.56 

9.22 

Student-Level Descriptive Statistics 

: Social Studies (Level 1) 


Female 

3,268 

0.52 

0.50 

African American 

3,268 

0.07 

0.25 

Hispanic 

3,268 

0.69 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

3,268 

0.70 

0.46 

TAKS Reading T score (2005) 

3,015 

49.02 

9.50 

TAKS Social Studies T score (2008) 

3,209 

48.49 

9.36 

School-Level Descriptive Statistics I 

(Level 2) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 
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Table E.18. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 2 


TAKS Achievement Test 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t-value 

Reading 


Initial status (spring 2005) 

52.427 

0.510 

102.75*** 


Immersion dummy 

-0.315 

0.581 

-0.54 


School poverty 

-0.091 

0.014 

-6.69*** 


Eco. disadvantaged 

-5.664 

0.596 

-9.50*** 


Growth rate 

0.131 

0.121 

1.08 


Immersion dummy 

0.234 

0.151 

1.55 


School poverty 

0.010 

0.005 

1.95+ 


Eco. disadvantaged 

0.308 

0.105 

2.95** 

Mathematics 


Initial status (spring 2005) 

52.152 

0.572 

gi 21*** 


Immersion dummy 

-0.891 

0.653 

-1.37 


School poverty 

-0.052 

0.019 

-2.70* 


Eco. disadvantaged 

-4.623 

0.594 

-7 78*** 


Growth rate 

-0.230 

0.191 

-1.20 


Immersion dummy 

0.653 

0.253 

2.58* 


School poverty 

0.005 

0.008 

0.70 


Eco. disadvantaged 

0.179 

0.111 

1.61 

Science 


Base 

50.839 

0.627 

81.14*** 


Immersion dummy 

0.475 

0.728 

0.65 


School poverty 

0.033 

0.023 

1.41 


Female 

-0.036 

0.289 

-0.13 


African American 

-1.902 

0.782 

-2.43* 


Hispanic 

-1.484 

0.493 

-3.01** 


Eco. disadvantaged 

-1.611 

0.367 

-4.39*** 


Spring 2005 T score 

0.641 

0.023 

28.00*** 

Social Studies 


Base 

51.850 

0.961 

53.95*** 


Immersion dummy 

0.006 

0.994 

0.01 


School poverty 

0.030 

0.029 

1.03 


Female 

-2.049 

0.304 

-6.75*** 


African American 

-1.938 

0.737 

-2.63** 


Hispanic 

-1.058 

0.518 

-2.04* 


Eco. disadvantaged 

-1.616 

0.492 

-3.29** 


Spr. 2005 reading T score 

0.537 

0.019 

27.88*** 


r p < .10; *p < .05; **p < .01; ***p < .001. 
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Table E.19. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 2 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 temporal variation 

26.4556 




Level-2 individual initial status 

52.6110 

3219 

11794.71 

0.000 

Level-2 individual growth rate 

0.8584 

3219 

3749.75 

0.000 

Level-3 school initial status 

1.7646 

39 

135.21 

0.000 

Level-3 school growth rate 

0.1302 

39 

118.03 

0.000 

Mathematics 

Level- 1 temporal variation 

19.6036 




Level-2 individual initial status 

56.7500 

3205 

15825.10 

0.000 

Level-2 individual growth rate 

0.7844 

3205 

3827.95 

0.000 

Level-3 school initial status 

2.9283 

39 

179.24 

0.000 

Level-3 school growth rate 

0.5526 

39 

429.52 

0.000 

Science 

Level- 1 student effect 

39.8845 




School mean 

5.8506 

39 

322.70 

0.000 

School pre-measure-outcome slope 

0.0114 

41 

106.45 

0.000 

Social Studies 

Level- 1 student effect 

50.4949 




School mean 

10.2243 

39 

368.23 

0.000 

School pre-measure-outcome slope 

0.0078 

41 

76.71 

0.001 


Cohort 3 (Seventh Graders) 

Table E.20. Descriptive Statistics for TAKS Reading and Mathematics, 
Cohort 3 


I Variable Name 

N 

Mean 

SD 1 

Repeated Measures Descriptive Statistics: Reading (Level 1) 

Time 

10,965 

1.00 

0.82 

TAKS Reading T score 

10,488 

48.80 

9.77 

Repeated Measures Descriptive Statistics: Mathematics (Level 1) 

Time 

10,965 

1.00 

0.82 

TAKS Mathematics T score 

10,549 

49.15 

9.37 

Student-Level Descriptive Statistics 

: Reading (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,655 

0.72 

0.45 

Student-Level Descriptive Statistics 

: Mathematics (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,655 

0.72 

0.45 

School-Level Descriptive Statistics: (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 
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Table E.21. Descriptive Statistics for TAKS Writing, Cohort 3 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

: Writing (Level 1) 


Female 

3,672 

0.51 

0.50 

African American 

3,672 

0.06 

0.23 

Hispanic 

3,672 

0.70 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

3,655 

0.72 

0.45 

TAKS Writing T score (2005) 

3.086 

50.42 

9.41 

TAKS Writing T score (2008) 

3,672 

49.15 

10.59 

School-Level Descriptive Statistics: (Level 2) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 


Table E.22. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 3 


TAKS Achievement Test 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t-value 

Reading 


Initial status (spring 2006) 

52.775 

0.533 

99.01*** 


Immersion dummy 

-0.751 

0.571 

-1.31 


School poverty 

-0.083 

0.017 

-4 73 *** 


Eco. disadvantaged 

-5.123 

0.750 

-6.83*** 


Growth rate 

-0.104 

0.198 

-0.53 


Immersion dummy 

0.105 

0.183 

0.58 


School poverty 

0.015 

0.006 

2.38* 


Eco. disadvantaged 

0.109 

0.171 

0.64 

Mathematics 


Initial status (spring 2006) 

52.557 

0.661 

79.46*** 


Immersion dummy 2 

-1.465 

0.669 

-2.19* 


School poverty 

-0.040 

0.019 

- 2 . 10 * 


Eco. disadvantaged 

-3.893 

0.660 

-5.90*** 


Growth rate 

-0.417 

0.216 

-1.93 + 


Immersion dummy 2 

0.787 

0.292 

2.69* 


School poverty 

0.025 

0.008 

3.05** 


Eco. disadvantaged 

-0.325 

0.198 

-1.64 

Writing 


Base 

51.390 

0.500 

102.75*** 


Immersion dummy 

-0.723 

0.579 

-1.25 


School poverty 

0.003 

0.017 

0.17 


Female 

-0.218 

0.278 

-0.78 


African American 

-1.099 

0.423 

-2.60* 


Hispanic 

-0.407 

0.542 

-0.75 


Eco. disadvantaged 

- 2.001 

0.339 

-5 91*** 


Spring 2005 T score 

0.644 

0.025 

25.61*** 


V < .10; *p < .05; **p < .01; ***p < .001. 

d Immersion students had significantly higher initial TAKS mathematics scores. A latent variable regression 
was run to control for the effect of this initial difference on the growth rate. The immersion effect was a 
significant predictor of the growth rate with and without controlling for initial differences. Thus, the 
coefficients from original growth model are reported here. 
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Table E.23. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 3 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 temporal variation 

27.9341 




Level-2 individual initial status 

56.5296 

3581 

24077.87 

0.000 

Level-2 individual growth rate 


Effect not random 2 


Level-3 school initial status 

1.7255 

39 

117.95 

0.000 

Level-3 school growth rate 

0.1450 

39 

78.03 

0.000 

Mathematics 

Level- 1 temporal variation 

23.0755 




Level-2 individual initial status 

56.2365 

3582 

28796.33 

0.000 

Level-2 individual growth rate 


Effect not random 2 


Level-3 school initial status 

3.4352 

39 

185.96 

0.000 

Level-3 school growth rate 

0.6815 

39 

217.18 

0.000 

Writing 

Level- 1 student effect 

54.5226 




School mean 

2.7112 

39 

123.22 

0.000 

School pre-measure-outcome slope 

0.0099 

41 

75.27 

0.001 


a Using chi-square to compare the deviance of the models with and without the individual growth rate being random 
showed that the addition of the individual growth rate being random resulted in a negligible contribution to the 
explanation of the outcome variance. 


Cohort 1 (Ninth Graders, Post-Immersion) 

Table E.24. Descriptive Statistics for TAKS Reading and Mathematics, 
Cohort 1 


I Variable Name 

N 

Mean 

SD 1 

Repeated Measures Descriptive Statistics: Reading (Level 1) 

Time 

16,555 

2.00 

1.41 

TAKS Reading T score 

15,777 

48.91 

9.68 

Repeated Measures Descriptive Statistics: Mathematics (Level 1) 

Time 

16,555 

2.00 

1.41 

TAKS Mathematics T score 

15.806 

49.28 

9.45 

Student-Level Descriptive Statistics: Reading (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

3,311 

0.71 

0.45 

Student-Level Descriptive Statistics 

: Mathematics (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,311 

0.71 

0.45 

School-Level Descriptive Statistics: (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.52 

16.83 
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Table E.25. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 1 


School-Level 

TAKS Achievement Test Analysis 

Gamma 

Coefficient 

Standard 

Error 

t-value 

Reading 

Initial status (spring 2004) 

53.283 

0.654 

81.430*** 

Immersion dummy 

-1.205 

0.730 

-1.650 

School poverty 

-0.072 

0.016 

-4.451*** 

Eco. disadvantaged 

-6.271 

0.586 

-10.707*** 

Growth rate 

-0.014 

0.088 

-0.158 

Immersion dummy 

0.203 

0.105 

1.932 t 

School poverty 

0.008 

0.003 

2.902** 

Eco. disadvantaged 

0.367 

0.084 

4 371*** 

Mathematics 

Initial status (spring 2004) 

52.372 

0.729 

71.820*** 

Immersion dummy 

-1.122 

0.883 

-1.270 

School poverty 

-0.046 

0.020 

-2.307* 

Eco. disadvantaged 

-4.523 

0.501 

-9.029*** 

Growth rate 

0.221 

0.128 

1.726 1 ' 

Immersion dummy 

0.313 

0.196 

1.602 

School poverty 

0.008 

0.006 

1.398 

Eco. disadvantaged 

-0.126 

0.118 

-1.073 


< .10; *p < .05; **p < .01; ***p < .001. 


Table E.26. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 1 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 temporal variation 

26.3757 




Level-2 individual initial status 

59.4712 

3193 

14295.82 

0.000 

Level-2 individual growth rate 

0.3824 

3193 

3656.23 

0.000 

Level-3 school initial status 

3.8714 

39 

184.90 

0.000 

Level-3 school growth rate 

0.0589 

39 

96.98 

0.000 

Mathematics 

Level- 1 temporal variation 

19.6943 




Level-2 individual initial status 

60.7862 

3189 

18778.21 

0.000 

Level-2 individual growth rate 

0.6443 

3189 

4181.73 

0.000 

Level-3 school initial status 

6.5953 

39 

240.93 

0.000 

Level-3 school growth rate 

0.3645 

39 

331.65 

0.000 
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